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Preface 


Integration of advanced technologies into the cockpits of new aircraft designs is 
an ever-growing concern of the United Air Force policy makers. New technologies bring 
extra responsibilities and add new challenges to the list of the tasks which USAF pilots 
are expected to master. As technology advances, so does the level of cockpit workload. 
To assess in-flight workload in combat or in a high threat environment, no proper 
measuring technique has been found. This study investigates pilot perception of cockpit 
workload in a high threat combat environment by collecting subjective opinions of USAF 
pilots. 

This study surveyed the attitudes of pilots who had flown different types of 
aircraft in a combat to provide insights into workload measurement in combat. A total of 
219 Air Force bomber, fighter, transport, air-to-air refueling, and tactical attack pilots 
with previous combat experience were surveyed. Respondents identified combat 
workload items with varying degrees of importance ranging from “distractingly” 
important to “a little or not” important. Overall, “distractingly” important combat 
workload items were similar for almost all types of aircraft as well as for most mission 
types flown in combat. Regardless of aircraft type or mission flown. Threat Avoidance 
and In-flight Emergency were dominant combat workload items. Pilot ratings for 
significant combat workload items were summed to assess the perception of in-flight 
workload of pilots on a combat mission. Means of Combat Workload Scores (CWL) for 
all aircraft types were not found to be significantly different. The study concluded that 
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combat workload can be measured by surveying the perceptions of pilots; however, the 
findings, based on subjective assessments, provide tentative grovmds for further research. 

Hereby, I would like to express my well-deserved thanks to special people who 
contributed to this research effort. Without them, this study would not have been 
completed. I would like to express my special thanks to my thesis advisor. Dr. David 
Kirk Vaughan, who guided my research effort from day one. He kept me on the track 
and helped me solve the formidable problems which I had as an international student. I 
also owe special thanks to Dr. Guy Shane for his help and guidance in assuring the 
accuracy of statistical analysis. 

Second, I want to thank the pilots of445“’ Airlift Wing at Wright-Patterson AFB, 
Ohio, and those of the Turkish Air Force, who participated in Red Flag in August of 
1997. They reviewed the preliminary survey, and gave insights to shape the final format 
of the questionnaire. Next, I owe special thanks to my friends who helped personally 
with organizing and mailing out the survey packages. Their dedicated work saved 
considerable time and effort. 

Finally, I want to thank my colleagues in the United States Air Force for taking 
time to answer my questionnaire. Their responses made this research possible. I hope 
that I interpreted their responses accurately, and presented the results as fairly as possible. 
I believe the conclusions driven from their responses will have an impact on future 
aircraft design efforts and in-flight workload assessment research. 


Kadircan Kottas 
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Abstract 

This study analyzed the self-reported survey responses of 219 Air Force Pilots 
concerning their perceptions of workload assessment in a combat or a high-threat 
environment. The first objective of this study was to determine and compare the combat 
workload factors of varying importance in combat workload assessment by aircraft and 
mission type flown. The second objective was to examine the pilots’ perception of 
combat mission in-flight workload. A stepwise regression model to predict the pilots’ 
perceptions of in-flight workload using pilots’ characteristics data was explored. 
Research conclusion varied among aircraft types. Combat workload items indicated as 
“distractingly” important were similar for all aircraft types, while items in lower level of 
importance were impacted by aircraft type. Mean Combat Workload (CWL) scores of 
pilots fi-om each aircraft type were not significantly different. Overall, it was concluded 
that surveying pilots who had flown in combat or high-threat environments provided 
useful responses to assess pilot workload; however, findings based on subjective 
assessments, provide tentative grounds for further research. 
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USAF PILOT PERCEPTIONS OF WORKLOAD ASSESSMENT IN A COMBAT OR 


HIGH-THREAT ENVIRONMENT 

1. Introduction 

Chapter Overview 

This chapter contains general background information relating to issues of in¬ 
flight workload measurement of pilots in combat environment. The specific problem 
statement, the purpose of the study, and investigative questions are presented as well as 
the associated hypotheses. Also included in this chapter are the assumptions of the study. 


Background 

Air operations depend on the aircrew's ability to employ all the capabilities of 
their weapons systems. However, with advanced weapons systems, aircrew tasks are 
becoming increasingly complex and time constrained, often near the limits of the 
aircrew's ability to perform required tasks. Faced with handling several sources of 
information concurrently, the pilot must decide on priorities. This becomes vitally 
important in a combat environment, where often an enemy threat is present. 

When the pilot fails to prioritize concurrent tasks, he or she becomes task 
saturated. Often, primary task capabilities, such as the ability to keep the desired flight 
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path, are impaired due to task saturation, perhaps resulting in unsafe situations. If this is 
the case, flight training emphasizes the importance of flying the aircraft first, with 
navigation and communication as secondary responsibilities (Edwards, 1990:190). 

Task saturation and workload are closely related. Once the pilots have reached 
the limit of their mental capacity, they start to exert less control over the tasks and 
activities in the cockpit. The pilots are subject to mainly two kinds of workload while 
flying, physical workload and mental workload. Physical workload involves G forces, 
heat effects, air pressure effects, body and head movements, and other physical tasks like 
changing the position of switches. Often, these tasks are easier to measure by using 
physical scales such as heart rate, breathing rate, muscle tension, EKG, eye and eye lid 
movement. On the other hand, mental workload is not so easy to measure. Williges and 
Wierwille remark that two problems are apparent from a review of literature pertaining to 
mental workload: first, there is a lack of a single agreed-upon definition of mental 
workload, and second, there was is lack of a universal scale to measure it (Williges and 
Wierwille, 1979: 549). Harper and Cooper suggest that the two most common elements 
of pilot workload are: (1) What the pilot is required to accomplish with the aircraft and 
(2) The conditions or circumstances under which the required operation is to be 
conducted (Harper and Cooper, 1984: 11). On the other hand, Edwards simply describes 
workload as “the demand on the pilot” (Edwards, 1990:133). 

As aircraft systems become more complex, system performance and safety 
become critical issues in crewstation design. Both system performance and safety are 
dependent on operator workload. Recognizing this fact, all three military services. 
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NASA, and the Federal Aviation Administration (FAA) have been developing and 
evaluating empirical measures of in-flight workload (Jensen, 1989:240). The Air Force 
Studies and Analyses and Aeronautical Systems Division/Equipment Engineering are two 
Air Force divisions which have become concerned with pilot workload. They add that 
the need to evaluate a system’s potential effect on workload prior to the purchase of the 
system has become important, particularly in an age of high cost systems and reduced 
budgets (Groves and Kaercher, 1981). 

Researchers have spent extensive effort and time on mental workload 
measurement studies. Some researchers, such as Wierwille et al in 1987, Moray 1982, 
and Gawron et al in 1989, updated the list of workload measures by reviewing the 
literature over time. They agree on four types of methods of measuring aircrew 
workload. They are: (1) Primary task performance measures, (2) Secondary task 
performance measures, (3) Physiological measures, and (4) Subjective measures. 

Most of the measuring techniques utilize testing environments including different 
types of test batteries, flight simulators, or rotary or fixed wing aircraft. They used 
student or experienced pilots as well as non-pilot subjects, depending upon the resoiffce 
availability or the purpose of the study. Use of test batteries is easy and inexpensive, 
while simulation requires more expertise from the stand point of both researcher and the 
subjects. Although simulation creates a more realistic environment in comparison to test 
batteries, simulators cannot provide an environment for most of the real world 
dimensions of pilot workload. For instance, pilot workload situations created by flying 
close to the groimd on low level missions or by maneuvering to avoid a deadly enemy 
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missile are difficult to recreate in simulators. Real world measurements carried out on 
aircraft are expensive and risky, because measurement techniques used might adversely 
impact the mission success. 


Problem Statement 

Flying the high performance aircraft of today is probably one of the most 
challenging tasks that humans are given. Flying these aircraft requires high levels of 
physical and mental skills, particularly in a combat environment where the number of 
tasks can increase drastically. Measuring the physical and mental workload is vitally 
important to determine both the physical and cognitive capabilities of aircrew so that 
these limits are not exceeded, and new systems are designed accordingly. However, 
measuring of pilot workload in a combat environment by using some kind of a test 
battery in a rotary or fixed wing aircraft is not feasible. These measuring methods will 
degrade the pilot’s performance or, at least, they will interfere with the flying tasks, 
where each task is extremely important to accomplish a combat mission. Using 
simulators as a testing environment for pilot workload measurement does not produce 
realistic results. A survey of pilots who have previous combat experience of any type 
should reveal a useful workload matrix in a combat environment with an existent enemy 
threat. Through such a survey, important aspects of in-flight workload assessment and 
mferences about pilot workload in high threat environments could be obtained. Findings 
of the survey might give insight regarding the efforts to measure workload in combat. 
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Research Question 


The overall question that this research is trying to answer is: Based on the pilot 
experience, is it possible to assess pilot workload in a high threat combat environment? 

The study is intended to investigate the pilot workload in a high-threat combat 
environment by collecting subjective opinions of USAF pilots currently flying AO/A-10, 
B-52, C-130, F-15, F16, and KC-135 aircraft. The results of the survey should provide 
insight to those interested in researching workload measurement in a combat environment 
without interfering with mission tasks, and to those willing to design operator systems 
that reduce the increased workload in the cockpit of a combat aircraft. 

To answer the research question, the following specific questions and hypotheses 
are investigated. 


Investigative Questions and Relevant Hypotheses 

1. What pilot workload measurement systems have been developed that might 
pertain to a high-threat combat environment? 

2. Is it possible to determine, from experienced pilots, what items would be most 
important in measuring workload in a high-threat environment? 

3. Can any kind of consensus be reached to determine the relative priorities of 
workload factors? 
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3a. What do pilots of different aircraft type believe are the critical items that 
would increase their workload in a high threat combat environment? 

Hypothesis #1: Pilots’ perceptions of each eiircraft type pertaining to the 
combat workload items will be different due to different characteristics of 
aircraft types. 

3b. Does the level of workload perceived by pilots change depending on the type 
of the mission flown? 

Hypothesis #2: The combat workload items’ ratings of pilots will vary 
depending on the type of the mission flown. 

3c. Does the level of workload perceived by pilots of each type vary by the 
experience level of the pilots? 

Hypothesis #3; Perceptions of combat workload of pilots with more flight 
experience will be likely to be lower than those of inexperienced pilots. 

3d. Is there a particular combat workload item that possesses significantly higher 
perception of workload by all pilots? 

Hypothesis #4: The workload of “Threat Avoidance” is perceived 
significantly higher than other combat workload items by all pilots. 

4. Is it possible to enumerate combat workload perceptions of pilots overall, or 
those of pilots from different aircraft, depending upon their characteristic 
data? 

Hypothesis #5: A model could be developed to predict combat workload 
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perceptions of pilots in general, or those of pilots from particular aircraft 
by using pilots’ demographic data as independent variables. 


Assumptions 

This study assumes that the pilots selected to participate in this survey represent 
an appropriate sample of the population of interest. It is also assumed that these 
individuals freely participated in the survey and provided honest answers to the questions. 


Summary 

This chapter has covered the general aspects of the workload measurement. The 
statement of the problem was provided in addition to the research question. Further, 
investigative questions were presented to detail the research question. In the next 
chapter, the published material pertaining to the in-flight workload measurement is 
investigated in detail. 
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II. Literature Review 


Chapter Overview 

This chapter reviews the published studies of researchers involved in efforts to 
measure the workload of operators such as pilots, space crew, and air traffic controllers. 
In this particular chapter, the various types of measurements are discussed according to 
their type and ability to measure in-flight workload. The review focuses mainly on those 
studies which are applicable to aircrew operating any type of Air Force rotary or fixed- 
wing aircraft. 

The review of the existing literature includes four major types of workload 
measurement techniques: 

1. Primary Task Performance Measurement Techniques 

2. Secondary Task Performance Measurement Techniques 

3. Physical Measurement Techniques 

4. Subjective Measurement Techniques 

Each technique is briefly described in the discussion that follows. Also in this 
chapter, the discussion of the purpose of the workload measures is included as well as the 
different methods described in the studies. Furthermore, the feasibility of measuring 
workload by using the listed workload measures is discussed in this chapter. Three 
studies which researched different aspects of aircrew flying issues during wartime 
operations by using similar or different methods were analyzed in terms of their relevance 
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to workload measurement in a high-threat setting. Finally, a summary of the findings 
gathered from the literature review is presented at the end of the chapter. 


Measurement of In-flight Workload 

As stated earlier, there is no single definition of aircrew workload and no single 
best way of measuring it. This is due to multi-dimensional feature of aircrew workload, 
which requires more than one type of measure to evaluate various aspects. The 
researchers agree that the appropriateness of a certain type depends on the purpose of the 
study. 

A recent review of the related literature by Gawron, Schflett and Miller (1989) 
states that there are three uses for workload measurement. The first use predicts the 
workload of a particular system configuration for design purposes. The second use 
evaluates workload demands of existing systems for comparing systems or other 
workload measures, and for scheduling purposes. The third use of workload measurement 
is the online monitoring of workload. Such monitoring can be used in ways such that 
excessive workload could be reduced by restructuring the tasks performed by the pilot 
(Gawron et al. 1989: 241). 

Several investigators have developed guidelines for selecting a proper workload 
measure for the purpose of the study. The most recent of guidelines, developed by 
Sequitur Systems, Inc. in 1988, includes the following seven criteria: 
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1. Sensitivity: the degree that a technique can detect changes in the amount of 
the workload imposed by the given task. 

2. Diagnosticity: the extent that a technique can identify the human capabilities 
being used to perform the task 

3. Intrusiveness: the extent that a workload measure degrades primary task 
performance. 

4. Implementation Requirements: the practical constraints associated with the 
complexity of the measurement procedures or the apparatus. 

5. Operator acceptance. 

6. Reliability: the consistency of the data or results 

7. Validity: the classical psychometric concept of validity including face, 
construct, content and predictive validity. 

The review of the written materials indicates that not only is there no single 
agreed definition of workload, but also there is no single agreed categorization of 
workload. Some researchers name the categories differently. For instance, Wierwille and 
Connor (1983) group them under the following categories: opinion, spare mental 
capacity, physiological, eye behavior and primary task techniques. However, researchers 
have agreed upon the four types of workload measurement techniques mentioned earlier. 


The Primary Task Performance Measurement Techniques 

In a 1993 study, Wierwille and Eggemeier state that these type of techniques 
measure the operator’s capability to perform the primary systems functions. Data about 
the operator’s performance are typically obtained in a test and evaluation environment, 
and can provide an index of operator workload. They add that when such primary task 
measures are used to evaluate workload, techniques are applied with the expectation that 
the speed or accuracy of performance will decrease as workload increases beyond critical 
value or threshold for unimpaired performance (Wierwille and Eggemeier, 1993: 268). 
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Gawron et al (1989) list three types of primary performance techniques: Pilot 
Performance Index, Task Load, and Time Limits. 

The Pilot Performance Index (PPI), developed by Stein in 1984, contains a list of 
performance variables and associated performance criteria for an air transport mission. 
The items in the list can distinguish experienced and novice pilots from each other. It is 
stated that the PPI is unable to measure the pilot performance effectively. 

Task Load is defined as the time required to perform the task divided by the time 
available to perform the task. Any value above 1 indicates excessive workload. Gawron 
notes that this technique is sensitive to workload in flight environments. 

The concept of Time Limits was developed by Gawron to identify dangerous 
situations and the performance of the pilot during these periods. These times include: 
time imtil impact, time until fuel is depleted, time until aircraft explodes or breaks apart, 
and other crisis-linked time limits. 


The Secondary Task Performance Measurement Techniques 

Secondary task scores were suggested as an index of workload and difficulty of 
certain primary tasks by Knowles in 1963. A study by Ogden, Levine, and Eisner (1979) 
states that the secondary task is used to determine how much additional work the operator 
can handle while still performing the primary job without deviating from desired level of 
performance. The authors note that this technique is also called as subsidiary task or 
auxiliary task (Ogden et al, 1979: 530). In this type of measurement, the subjects are 
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given two tasks concurrently, and asked to perform the auxiliary test giving emphasis on 
the primary task. The primary task might be determined or changed during the. 
experiment by instructions from the experimenter. 

Use of this technique involves the comparison of the performance of the 
secondary and the primary task together and that of solely the secondary task. The 
difference in performances indicates the magnitude of the workload imposed by the 
primary task. This type of measurement is based on the assumption that the secondary 
task does not interfere with the primary task performance. Another use of the technique 
includes the determination of the spare mental capacity, in which the performance of the 
subject on the secondary task indicates a scale for the difficulty of the primary task; the 
less the spare mental capacity the pilots have, the more the workload they experience. 

Gawron et al list the five major types of secondary workload measurement. These 
include: Mental Arithmetic, Vigilance, Tracking, Time Estimation, and Sternberg Test. 

In the Mental Arithmetic technique, pilots are asked to perform basic arithmetic 
operations on sets of visually or aurally presented numbers while performing a primary 
task such as navigating, tracking, or piloting. The Vigilance technique requires pilots to 
respond either manually or verbally to the onset of visual or auditory stimuli. The 
Tracking tasks require the pilot to nullify the error between a desired and an actual 
course. The authors note that the Mental Arithmetic technique is useful when continuity 
is desired in workload measurement, and it is a proper technique when experimenting 
with simulators. 
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In the Time Estimation technique, pilots are asked to estimate a certain time 
interval (usually 10 seconds) starting with a visual or an aural stimulus while performing 
piloting tasks. The number of incomplete estimates or the length of the estimates might 
be used as an index of the workload of the primary task. The Sternberg Test requires 
pilots to determine if a given letter is a part of a previously memorized set of letters. The 
reaction times are measured at two different memory sizes (usually two- and four-letter 
sets) and plotted on a chart. The differences among the slopes of the various design sets 
indicate the differences in information processing demands. 

Gawron et al remark that the major disadvantage of measuring secondary tasks is 
intrusion into the primary task, which is not desired in workload measurement. Besides, 
intrusion caused by a secondary task reduces the operator’s acceptance of that 
measurement test dramatically. 

After reviewing the post-1965 literature on the use of secondary tasks in 
assessment of the operator workload, Ogden et al. found twelve classes of tasks were 
used as secondary tasks. The most frequently used secondary tasks were choice reaction 
time, memory, monitoring and tracking. The other tasks included reaction time, 
classification, navigation, motor response, shadowing, detection and mental math. 
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Physiological Measurement Techniques 


One of the most widely used methods of assessing pilot workload is use of 
physiological measures. Wierwille states that the underlying concept in physiological 
variables is as follows: 

As operator workload changes, involimtary changes take place in the 
physiological processes of the human body (body chemistry, nervous system 
activity, circulatory or respiratory activity, etc.). Consequently, workload may be 
assessed by the measurement and the processing of the appropriate physiological 
variables. (Wierwille, 1979: 575) 

Wierwille adds that this technique assumes that changes in the workload cause 
increased emotional stress which is an intermediate variable causing physiological 
changes in the human body. 

Gawron et al (1989) list the following nine physiological measurements of 
workload: Heart Rate, Heart Rate Variability, Weighted Coherence and Vagal Tone, 
Blood Pressure, Ventilation and Metabolic Rate, Spontaneous Electroencephalogram, 
Evoked Potentials, Endogenous Blinks and Visual Activities. 

Heart Rate is typically measured by the number of the beats of the pilot’s heart 
per minute. Measures are taken by electrocardiographs, as well as manual recordings of 
physical pulse rates. This measure has been used to compare the workload of pilots on 
different segment of the flights as well as among pilots of different aircraft types. Heart 
Rate Variability (HRV) is measured by the variability of interbeat intervals of the heart 
rates. Gawron et al note that HRV is affected by the breathing process. Weighted 
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Coherence and Vagal Tone techniques stem from the frequency analysis of HRV, and 
these techniques are reported to have applications in aviation psychology. 

Blood Pressure is the measure of the body fluid’s pressure exerted against the 
inner walls of the vanes. It is measured in units of inches or millimeters of mercury. 

This technique is used only for pre- or post-flight measurements, and the difference might 
provide an index for in-flight workload; however, it is affected heavily by the emotional 
status of the pilot and environmental factors like temperature and acceleration forces. 

Ventilation and Metabolic Rate of lungs can provide various measures like 
ventilatory minute volume and breathing (ventilation) rates. According to Gawron et al, 
studies have proven that Spontaneous Electroencephalogram (EEG) (brain electrical 
signals) measures can provide a scale for pilot consciousness during in-flight operations. 

Evoked Potentials are described as the changes in the amplitudes and latencies of 
cortical responses to external stimuli. It is reported that this measure is used minimall y in 
flight due to difficulties of introduction of a well-defined stimulus into the task scenario 
(Gawron et al, 1989:256). 

Endogenous Blinks are different from exogenous blinks (reflex and volimtary 
blinks) as a result of the lack of an identifiable, external eliciting stimulus. Investigations 
of eye blink accompanied by event-recorded data could provide information about where 
the pilot’s attention is focused. Studies show that the number of blinks decreases as the 
in-flight workload increases. Several other measurement data can be derived by 
examining the different visual cues provided by the pilot such as elapsed time between 
fixation on primary performance instruments, time spent monitoring in-cockpit versus 
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out-of-cockpit activities, and movement of the eyeball (rate and frequency) (Gawron et al, 
1989: 257). 

In addition to the list above by GaAvron et al, in a review of physiological 
measurement techniques of mental workload, Wierwille (1979) lists the following 
measurement techniques: Flicker Fusion Frequency, Galvanic Skin Response and Skin 
Impedance, Electromyogram, Muscular Tension, and Body Fluid Analysis. 

Flicker Fusion Frequency (FFF) exposes the subject (pilot) to the lowest 
frequency of flickering light source where it is perceived as a constant light source 
instead. The Galvanic Skin Response (GSR) technique measures the resistance of the 
skin tissue of the human body to the flow of low-level electrical current. On the other 
hand. Skin Impedance measures the resistance of the epidermal (skin) tissues to 
alternating current. An Electromyogram (EMG) measures and records the electrical 
activity of the epidermis caused by the electrical potential of the muscle under the skin. 

Pilots often realize that various muscles in the body are tensed when the number 
of activities increases; for instance, leg and arm muscles’ tension increase during close 
formation flight. Wierwille states that muscle size changes also can be used as a measure 
of the workload due to fact that tense muscles increase in size; however, in-flight 
mezisurement might require advanced apparatus. 

Wierwille states that when a human being undergoes stress or becomes fatigued, 
changes may take place in the body’s metabolic balance which are manifested in various 
body fluid compositions. Since sustained workload can induce stress and fatigue, it may 
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be possible to relate workload to these changes of the body fluid such as urine or parotid 
fluid. 


Subjective Workload Measurement Techniq ues 

Subjective workload measurements are commonly based on subjective opinions 
of the pilots about the magnitude of the workload they are exposed to. Wierwille and 
Eggemeier (1993) suggest that current results indicate that subjective techniques can be 
generally classified as globally sensitive indexes of workload (Wierwille & Eggemeier, 
1993:267). Casali and Wierwille (1983) note that subjective measures have been widely 
used in pilot workload assessment due to their simplicity, flexibility, low cost and 
imobtrusiveness. Gawron et al (1989) list the following types of subjective workload 
measurement techniques; Subjective Workload Assessment Technique, Crew Status 
Survey, Profile and Mood States, Hart and Hauser Rating Scale, Flight Workload 
Questionnaire, NASA Bipolar Rating Scale, NASA Task Load Index, Analytic 
Hierarchy Process, Cooper-Harper Rating Scale, Bedford Workload Scale, Pilot 
Objective/Subjective Workload Assessment Technique, and Computerized Rapid 
Analysis of Workload. 

The Subjective Workload Assessment Technique (SWAT) combines three 
different subscales under one measurement index including time load, mental effort load 
and physical stress load. This technique requires the pilot to rank workload from lowest 
to highest for each level of subscale. Pearson and Byes developed the Crew Status 
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Survey, composed of 20 statements where each statement describes different fatigue 
states. Eventually the number of the statements was decreased to seven statements 
besides blank space for general comments to simplify the scale. The abstract version of 
Profile and Mood States (POMS), which was developed by Shachem in 1983, provides 
measures for anger, vigor, fatigue, confusion and self-rated stress (Gawron et al, 1989: 
260). 

The Hart and Hauser Rating Scale was developed in 1987 to measure the 
workload for sustained flights by using a six-item rating scale. The six items, which used 
related rating scales from low to high extremes, include stress, mental/sensory effort, 
fatigue, time pressure, overall workload, and mission performance. 

The Flight Workload Questionnaire utilizes a four-item behaviorally-anchored 
rating scale. Items include workload category (low to very high), fraction of time busy 
(seldom have much to do to fully occupied all times), difficulty of thinking/concentration 
( minimal thin kin g to a great deal of thinking), and state of feeling (relaxed to very 
stressful) (Gawron et al, 1989: 263). 

The NASA Bipolar Rating Scale has ten subscales with subjective associated 
rating scales. Subscales consist of overall workload, task difficulty, time pressure, 
performance, mental /sensory effort, physical effort, frustration level, stress level, fatigue, 
and activity type (skill based, knowledge based and rule based). The NASA Task Load 
Index (TLX) is a multi-dimensional subjective workload technique in which workload is 
defined as efforts given by the pilot to achieve a certain level of performance. Subjective 
responses of pilots include emotional, cognitive and physical reactions as well as 
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weighted evaluations of their behaviors. The scales, with mostly low to high rating, 
include mental demand, physical demand, temporal demand, performance, effort, and 
fhistration level (Gawron et al, 1989:264). 

The Analytical Hierarchy Process (AHP) asks pilots to compare all possible pair 
combinations to determine which condition has the higher workload. The Cooper-Harper 
(C-H) Rating Scale determines the handling qualities of newly designed aircraft systems, 
specifically, as used by the test pilots. Wierwille and Casali (1983) modified the original 
C-H rating scale to be able to assess the workload of the pilot. The Bedford Workload 
Scale is another version of the Modified C-H Rating Scale, which was developed by 
Royal Aircraft Establishment at Bedford, England. The Bedford Workload Scale uses the 
same binary-type rating scale with three-, four-, and ten-rank ordinal structure. The Pilot 
Objective/Subjective Workload Assessment Technique (POSWAT) is a ten-dimensional 
scale adopted from the modified C-H rating scale. It was developed by FAA Technical 
Center by using five categories of ratings as opposed to binary rating scale of the 
Modified C-H Rating Scale. The Computerized Rapid Analysis of Workload (CRAWL) 
is mainly used to assess the workload in systems being designed (Gawron et al, 1989: 
265). 


Feasibility of the Workload Measurement in Hieh-Threat Environment 

Almost all of the measurement techniques discussed previously require a test and 
evaluation environment or an advanced set of apparatus to collect data for evaluation. 
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These techniques generally impose intrusions into the primary mission of the operator. 
The subjective measures, even though the least obtrusive type of measurement 
techniques, require some kind of stiff surface board for the responses to be written on by 
the pilots. Some studies used audio recordings; nevertheless, they still cause degradation 
in primary mission performance. 

The extensive review of the written materials on workload assessment reveals that 
there have been only a few studies done during real flight operations. A study by Wilson 
(1993) performed psycho-physiological workload analysis on the physiological and 
behavioral data collected on an air-to-ground mission on F-4 Phantom aircraft (Wilson, 
1993:1071). Another study by Skelly, Purvis, and Wilson used a physiological 
technique which collected three t3q)es of data (heart (ECG), eye (EOG) and brain (EEG) 
electrical signals) by use of pocket-carried physiological tape recorders (Gawron et al., 
1989: 257). Only a few other in-flight workload assessment studies were performed on 
transportation and airborne refueling aircraft or rotary-wing aircraft, where the 
researchers had comfortable space for the apparatus to be utilized, as in the study 
conducted by Hill et al (1992). Numerous studies can be fotmd in the literature which 
used simulators or test batteries in workload measurement. Simulators and test batteries, 
used individually or together, provide opportunities whereby various experimental 
designs in workload measurement can be implemented easily . 

These facts strongly indicate that these measures will not be applicable to real 
world workload assessments endeavors in a combat environment where an enemy attack 
is imminent. This is mostly due to intrusion problems of the measiirement techniques 
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and mission effectiveness concerns. A study by Neville et al, which is discussed in the 
next section in more detail, concludes that, unfortunately, no suitable methodology for 
measuring pilot performance in a wartime setting could be found (Neville et al, 1994: 
346). Use of advanced simulators could be considered to assess the workload in such an 
environment; however, the effects of real threats are almost impossible to create in a 
simulation world. 


Related Studies 

Three previous studies researched different aspects of aircrew flying issues 
during wartime operations. However, they used similar or somewhat different 
methodologies to probe the issues. I have foimd it beneficial to include the discussions of 
these studies to point out the important issues mentioned in them. These studies include a 
quasi-experimental study of pilot fatigue in Desert Shield and Desert Storm by Neri and 
Shappell (1992), another quasi-experimental study by Neville et al (1994) that also 
investigated effects on fatigue during Operation Desert Storm, and an AFIT master’s 
thesis by Starr and Welch (1991) which surveyed the USAF pilots’ attitudes regarding 
replacing Weapon System Operator (WSO)/Navigator/Electronic Warfare Officer (EWO) 
with new cockpit technologies. 

Neri and Shappell (1992) 

Neri and Shappell investigated the effects of combat on the work/rest schedules 
and fatigue of 23 naval aviators aboard the U.S.S. America during Operations Desert 
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Shield and Desert Storm. They collected daily activity logs and recorded subjective 
fatigue measures dxiring a four-week period while aviators operated in the Red Sea 
combat theater. They probed the indications of fatigue and sleep disorders induced by 
combat operations. 

Neri and Shappell used a six-item survey card which volimteer pilots and radar 
intercept officers of F-14 Tomcat and A-6 Intruder aircraft aboard the carrier filled out 
after completing a day or night mission. Data collection started as soon as the U.S.S. 
America departed Norfolk, Virginia, on December 28,1990 for the Red Sea. Operation 
Desert Storm started in the morning of January 16, just the next day after the arrival of 
the carrier in the theater. The researchers gathered the two-week data while the carrier 
passed seven time zones during her cruise. They also collected two-week data when the 
carrier anchored in the Red Sea combat theater during Operation Desert Storm. 

They investigated the sleep quality of pilots and possible effects of combat such 
as sleep disorders. They also examined the effects of passing several time zones in a 
short time period on the sleep patterns of the aviators during and in the aftermath of the 
cruise. They claimed that studying the issue would help the collection of valuable 
information about military sleep management. 

Neri and Shappell found that the aviators flew frequently at night without 
significant sleep problems or fatigue. They claimed that a likely factor was the excess 
amount of military assets in the combat theater, which allowed the workload to be shared 
by other units. The investigators pointed out other contributing factors, such as the effect 
of the eastward travel on pilots’ circadian pacemakers. They stated that the inability to 
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adapt to the seven-time zone-change allowed pilots to fly the night missions around 0300 
hours local time (2000 EST) without significant sleep problems. Neri and Shappell 
concluded that the work and rest schedule of critical combat personnel such as certain 
aviators. Special Forces’ members, and advanced weapon operators could be revised to 
benefit from the side effects of circadian desyncronization (Neri and Shappell, 1991: 

160). 

This particular study is unique in studying the effects of combat on pilot 
performance in the real world environment. Similar studies could be performed to 
investigate the workload of the pilots in a real combat environment. The possibility of a 
major regional conflict like Operation Desert Storm occurring is hard to determine and it 
is highly undesired. Nevertheless, investigators should not disregard the future 
opportunities to assess the workload of pilots during wartime operations. 

Neville. Bisson. French. Boll and Storm fNeville et al. 19941 

Like Neri and Shappell (1992), Neville et al (1994) investigated the effect of pilot 
fatigue on mission performance during Operation Desert Storm. Neville et al collected 
subjective fatigue ratings of eleven C-141 pilots from the 437* Military Airlift Wing, 
Charleston AFB, South Carolina, by using School of Aerospace Medicine (SAM) fatigue 
scale and Profile of Mood States (POMS) in addition to Digital Flight Data Recorder 
(DFDR) performance data. Data collection included a time span starting on March 16, a 
week prior to end of war, and ending on April 11,1991. 
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On the SAM fatigue scale card, aircrew record subjective perception of fatigue 
experienced on a scale ranging from 1 to 7. The scale consists of following ratings: 1) 
fully alert, wide awake, very peppy; 2) very lively, responsive, not at peak; 3) okay, 
somewhat fresh; 4) a little tired, less than fresh; 5) moderately let down; 6) extremely 
tired; and, 7) completely exhausted, xmable to function. The card also asks information 
about caffeine consumption, location of mission, and activities on half-hourly basis. 
Subjects recorded fatigue ratings and oral temperature every four hours, unless they were 
asleep. The other subjective fatigue measure used in the study, POMS, was developed by 
McNair, Lorr, and Droppleman (1971). The POMS, which consists of 65 adjective 
ratings, were used to record crews’ perception of fatigue at the beginning of each crew 
rest period and the “legal-for-alert” period (Neville et al, 1994: 342). Ninety-six digital 
flight data recordings, obtained by DFDR, containing flight performance data on 138 
approaches, were used in analysis, 78 of which were ILS approaches. 

To evaluate the effectiveness of crew rest, Neville et al compared the POMS taken 
at two different times by performing an analysis of variance (ANOVA). In the analysis 
of aircraft performance, an 80-second segment of the last 90-second portion of each 
approaches is evaluated, excluding the last 10-second segment in which pilots 
transitioned to visual cues for touchdown. The mean indicated airspeed (IAS) readings 
within the 80-second period were averaged to analyze performance data. Addition to 
IAS, heading deviations were analyzed to evaluate the performance of pilots. 

Neville et al report that analysis of POMS indicates that aircrew members were 
able to recover from the high levels of fatigue at the start of crew rest. They add that 
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crew members suffered high levels of fatigue at the end of missions and possibly during 
the final few hours of missions. Neville et al found that recent flight and sleep histories 
are correlated with high levels of fatigue. Analysis of flight performance data revealed 
that aircrew members experiencing higher levels of fatigue have tendency to make more 
mistakes. 

Like the study by Neri and Shappell, this particular study pursues a similar 
method in studying the effects of combat on pilot performance in the real world 
environment. Use of DFDR to evaluate the pilot performance could be adopted to assess 
the pilot workload in a combat environment; however, the data should be supported by 
simultaneous cockpit aural or visual recordings to verify performance deviations. 

Possible correlation between workload and flight performance could be evaluated by 
using such recordings. 

Starr and Welch ri99U 

Starr and Welch surveyed 404 U.S. Air Force pilots regarding their perceptions of 
replacing the WSU/Navigator/EWO with advanced cockpit technologies. They also 
examined the impacts of such replacement on combat mission effectiveness. Their 
survey analyzed the responses of pilots from A-10, B-52, C-130, F-15E, F-16, and KC- 
135 umts to investigate the possible differences in responses by the aircraft type. 

Their survey, which contained 80 items, consisted of two parts. The first part was 
developed solely to acquire the demographic data to differentiate the various experience 
levels of the respondents by aircraft type. The second part was developed to collect data 
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to accomplish the four goals of the study. The first objective was to compare mission 
effectiveness factors that are always critical and almost always critical to the success of 
the success of the mission. The second objective was to examine, fi-om the pilots’ point 
of view, the WSO/Navigator/EWO impact on combat mission effectiveness (NAVCRIT). 
The third objective was to examine the WSO/Navigator/EWO’s contribution to overall 
combat mission effectiveness (REQ) 

Starr and Welch determined 31 mission effectiveness factors presented in the 
second part of their survey through the review of the related literature and interviews with 
the authorities. Pilots were then asked to rate each factor using a five-point Likert scale 
with the following choices: 1) Always Critical to Mission Success, 2) Almost Always 
Critical to Mission Success, 3) Can be Critical, 4) Almost Never Critical, and 5) Never 
Critical. Most of these factors, which are applicable to the purpose of the present 
research, were either rephrased or used directly. 

The researchers utilized surveyed pilot demographics to predict NAVCRIT and 
REQ by a stepwise regression model. NAVCRIT was a variable which measured the 
impact of a WSO/Navigator/EWO on the specific combat mission effectiveness factors 
selected as critical by a particular pilot. REQ was the variable researchers used to 
measure a pilot’s perception of the requirement for a WSO/Navigator/EWO for a specific 
mission. As the independent variables, they used the aircraft type, total flying hours on 
the current aircraft, total flying hours, previous aircraft qualification, flying experience 
with WSO/ Navigator/EWO, combat flying hours in the current aircraft, total combat 
flying hours, experience as an instructor pilot in an operational aircraft, experience as a 
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flight evaluator, experience as a wing weapon and tactics officer, participation in any 
major exercise, and the current rank of the respondents. 

Upon examination of NAVCRIT and REQ scores. Stair and Welch found that the 
pilots’ perceptions of WSO/Navigator/EWO contribution to combat mission effectiveness 
varied by aircraft type. Nevertheless, the “always critical” mission effectiveness factors 
were similar across all aircraft types. From the data gathered by reviewing the literature 
and surveying the 404 USAF pilots, the researchers concluded that the pilots of USAF 
aircraft did not believe it is possible to effectively replace the WSO/Navigator/EWO with 
technology for aircraft performing high threat combat missions. 

The importance of this particular study is twofold. First is the similarity of the 
methodology to that proposed in this thesis. Second is the close relationship between 
mission effectiveness factors and the combat workload factors that impact the pilot 
workload. The probable relationship between these factors could be observed by 
analyzing the survey results. 

Although the three studies detailed above investigate the some aspects of the pilot 
workload by including questions regarding the pilot workload, they do not suffice to draw 
significant inferences about the perception of the pilot’s workload in combat. This study 
intends to explore these factors to point out the probable problem areas of workload 
assessment of pilots in combat. 
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Summary 


There have been various attempts to measure the workload in flight which 
resulted in development of new techniques within last three decades. Investigators agree 
that there is no single best technique to measure workload. 

Selection of a proper workload measure depends on the purpose of the study. A 
1988 study by Sequitor Systems, Inc. lists the following seven criteria that should be 
considered while selecting a measure for workload assessment. These criteria are: 
sensitivity, diagnosticity, intmsiveness, implementation requirements, operator 
acceptance, reliability and validity. 

The workload measurement techniques are grouped imder four categories: (1) 
Primary Task Performance Techniques, (2) Secondary Task Performance Techniques, (3) 
Physiological Measurement Techniques, and (4) Subjective Techniques. Wierwille 
combines primary and secondary task performance techniques trader one name of 
“performance related measurement techniques.” 

Reviewing the material on assessment of pilots’ in-flight workload revealed no 
existing and suitable method of measuring workload in an aircraft while performing high 
threat combat missions. However, three studies researching different aspects of pilot 
operations in combat environment were discussed. 

Due to implementation requirements of the most measures, it is hardly feasible to 
use these measures in the real world environment. Specifically, the use of these 
techniques in a combat environment with existing enemy threats becomes almost 
impossible due to intrusiveness of the techniques and their impact on mission 
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effectiveness. Therefore, a survey of pilots with combat experience might reveal valuable 
information about workload in such an environment. 

In the next chapter, the methodology followed in this study is discussed, as well 
as the description of the survey questionnaire. 
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HI. Methodology 


Chapter Overview 

This chapter describes the methodology of the study including the survey 
instrument by which the data of the study were collected. It also includes the discussion 
regarding the survey construction and testing. Furthermore, this chapter covers the 
general description of the population and sample surveyed. Also in this chapter, 
reliability and statistical tests used in data analysis are described. 


Survey Instrument 

The survey questionnaire presented in Appendix A was used for data collection. 
To assure the anonymity of responses, questions about individuals’ name, social security 
number, and duty location were excluded from the survey questionnaires. Respondents 
were instructed not to put their names on any of papers to be returned. 


Survey Construction and Testing 

The survey questionnaire was designed in compliance with Air Force Instruction 
36-2601 (AFI36-2601, 1995:1) and the Graduate School of Logistics and Acquisition 
Operating Instruction 53-10 (LA/OI 53-10,1997:1). During the survey construction 
misleading and ambiguous questions were avoided as much as possible. Because the 
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survey questionnaire was designed to collect data from personnel flying different aircraft, 
each respondent was provided with an appropriate answer on each item. 

The survey the survey consisted of 53 questions and was composed of two main 
parts, personal characteristics and combat workload items. 


Personal Characteristics 

The first part of the questionnaire, personal characteristics, included eight 
questions about demographic and experience items of the respondents. Eight items 
collected information about respondents including current rank, gender, total flight hours, 
the current/most recent aircraft, the previous aircraft, combat flight hotus, aircraft and 
mission type flown in combat. These questions were asked to determine the experience 
level of each pilot. Questions regarding gender were asked to study the differences that 
might exist between the members of each group. 


Combat Workload Items 

The second part of the survey, combat workload items, consisted of two groups of 
questions. The first group of questions. Questions 9 to 37, contained 29 combat 
workload items. Pilots were asked to evaluate each item in terms of the degree to which 
they believed each item was likely to increase the pilot’s workload in a combat 
environment, using a five-point Likert scale. Most of the items were taken from an AFIT 
study by Starr and Welch (1991) but reworded for the purpose of this study. Other items 
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were added based on the flight experience of the advisor, the author, and other pilots 
during the individual reviews of the questionnaire in survey construction phase. 

The combat workload items were selected according to their potential impact on 
increasing the cockpit workload of the pilot in a combat environment. Maximum effort 
was made to select combat workload items applicable to all aircraft types included in the 
research. Nevertheless, due to fact that some of combat workload items might not apply 
for each aircraft type, the “not applicable” choice was included in the choices apart firom 
five-point Likert scale. By the help of this separate choice, those workload items which 
did not apply to the pilots of aircraft not performing certain tasks could be categorized 
separately. 

The twenty-nine combat workload items were listed without providing an 
operational definition. Many items conveyed different meanings depending on the 
aircraft type flown. These items were worded in a such way that the pilots filling out the 
survey could interpret the precise meaning of a particular item in their own operational 
terms. These items were listed in random order. 

Pre-test groups from two aircraft types were asked to evaluate the validity of the 
combat workload items. The complete list of the 29 combat workload items is presented 
below in the order that they appeared on the survey questionnaire. Those items taken 
directly from the previous questionnaire used in the masters’ thesis by Starr and Welch 
(1991) are indicated with an asterisk, and those items reworded for the purpose of the 
study are indicated by a double-asterisk. 


3-3 



1. Mission Planing* 

2. Terrain Avoidance/Following * 

3. Maintaining Situational Awareness** 

4. Adverse Weather * 

5. Monitoring Flight Instruments** 

6. Equipment Degradation ♦* 

7. Low Level Navigation * 

8. Night Low Level Navigation * 

9. Threat Avoidance * 

10. Formation Responsibilities ** 

11. Management of Time Over Target (TOT) * 

12. In-flight, No-Notice Mission Changes* 

13. Shifting Attention to Targets of Opportunity ** 

14. Munitions Employment * 

15. Threat Detection * 

16. Crew Incapacitation 

17. In-flight Emergency ** 

18. Visual Orientation ♦* 

19. Command & Control (such as copying & decoding EAMS)** 

20. Fatigue ** 

21. Crew Coordination ** 
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22. Aircraft Maiieuvering (Dogfight or avoiding the threats)* * 

23. Target Acquisition * 

24. Type of the Drop 

25. Night Operations * 

26. Unfamiliar Terrain 

27. Managing Radio Communication 

28. Refueling Operations 

29. Responding to Grovmd/Airbome Controller Instructions. 

The definition of each choice in the five-point Likert Scale indicating severity of 
workload distraction was provided in the beginning of the second part of the survey. A 
copy of the survey questionnaire including these definitions is provided in Appendix A. 
The choices of the five-point Likert Scale are listed below: 

1. Dangerous increase 

2. Distracting increase 

3. Moderate increase 

4. Some increase 

5. Little or No increase 

At the end of the list of combat workload items, the respondents were allowed to 
list additional workload items that they thought would be likely to increase pilot 
workload in a combat environment. 

The second group of questions. Questions 38 to 44, consisted of statements which 
asked pilots to evaluate these statements in terms of the degree to which they agreed with 
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them. All five questions were worded in such way that they might apply to all aircraft 
types. Questions 38,39,40,41, and 44 were used to obtain information about pilots’ 
perceptions on measures to reduce workload. The five questions are listed below: 

38. Additional aircrew could reduce the workload in a combat environment 
(Assuming that your aircraft could be designed in such a way to accommodate the 
additional aircrew such as Weapon Systems Officer (WSO), Electronic Warfare 
Officer (EWO), Navigator, or Radar Intercept Officer (RIO). 

39. Modifying the cockpit resources of your aircraft could help to eliminate the excess 
workload in the cockpit of the aircraft you flew in combat. 

40. Technological innovations in future aircraft designs will help reduce the 
workload. 

41. Updating the current operational regulations and procedures could help eliminate 
excess workload in the cockpit of the aircraft you fly or the most recent aircraft 
you flew. 

44. In combat, the superiority of the U.S. and allies’ air power over that of enemies 
will have an positive impact in reducing the amount of workload experienced 
during in-flight operations. 

In question forty-two, pilots were also asked for their opinions about the 
comparison of combat and peace-time pilot workload. Question 42 stated that the 
workload of combat flight operations is heavier than that of peace time operations. In 
Question 43, pilots were questioned about the effectiveness of simulator training. 
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Question 43 was stated as follows: Simulator missions flown in peace time adequately 
simulate the amoimt of workload that a pilot will experience in combat. 

For Questions 38-44, the respondents used a five-point Likert Scale to answer the 
questions in this group of questions. The scale included the following response choice; 

1. Strongly agree 

2. Agree 

3. Neutral 

4. Disagree 

5. Strongly Disagree 

Questions 44a through 45a solicited information regarding peacetime/pre-combat 
scenarios or procedures that pilots thought that would be most difficult to respond to in 
combat. They were also asked whether the scenarios or procedures were harder to react 
to in real combat situation. 

In the second group of questions in Part II of the survey questionnaire, pilots were 
asked to express their willingness to participate in any experimental studies that measure 
workload using data collected in flight in a combat environment and m simulated combat 
missions. Answers to this question could provide insights to future researchers. At the 
end of the survey, respondents were encouraged to make additional comments about the 
issues discussed in this survey. 
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Survey Pre-testing 


A preliminary version of the survey was administered to the following three 
groups: 1) pilots in the 445* Airlift Reserve Unit at Wright-Patterson AFB, OH; 2) pilots 
from several Turkish Air Force tactical fighter squadrons who attended the Red Flag 
Exercise in August 1997 at Nellis AFB, NV; 3) pilots attending the Air Force Institute of 
Technology (AFIT) Graduate School of Engineering. As result, fourteen F-16 pilots, 11 
C-141 pilots, and two F-15 pilots were pretested. 

Of the 40 surveys administered to the Turkish pilots, who all fly F-16s, 11 surveys 
were returned; and, of the 20 surveys administered to the 445* Airlift Reserve Unit, who 
fly C-141s, 11 surveys were returned. Even though individuals were encouraged to 
make any comments about the clarity, comprehensibility, and any problem areas of the 
survey questionnaire, no comments were made on any of the 22 surveys returned. 

Analysis of the twenty-two returned surveys did not reveal any problems. Some combat 
workload items were revised in accordance to survey comments made by pilots attending 
AFIT. 


Population 

In an effort to answer the main research question and investigative questions 
introduced in chapter one, pilots who had previously participated in combat, or flown in a 
high-threat environment in A-10, B-52, C-130, F-15, F-16, and KC-135, were surveyed. 
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These aircraft types were selected for several reasons. First, this listing contains 
aircraft which were used widely by the U.S. Air Force at the time of the survey. 

Including active Air Force, Air National Guard, and Reserve Units’ inventories, there 
were 274 A-lO/OA-lOs, 85 B-52s, 878 C-130s, 529 F-15s, 859 F-16s, and 645 KC-135s 
flying in the U.S. Air Force as of June 1997 (USAF Aircraft Fact Sheets, 1997:1). 
Therefore, flyers of these aircraft represent a good portion of USAF pilots. Second, each 
aircraft has different operational role and aircrew capacity requirement. Studying 
different aircraft types should give insights for a variety of cockpit workload assessments. 
Finally, all of the listed aircraft types were used in the Operations Desert Shield and 
Desert Storm. These aircraft flew several day- and night-time missions, and influenced 
the outcome of the war greatly. 

The total number of pilots currently flying each aircraft is listed in the table below 
(HQ AFPC/DPAO, 1997: 2). 


Table 3-1. Actively Flvins USAF Pilots 


Aircraft 

Total number 
Actively Flying 

A-10 

352 

B-52 

203 

C-130 

833 

F-15 

555 

F-16 

1084 

KC-135 

1157 
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Sample 


The sampling method used in the study is proportionate stratified sampling. Most 
populations can be segregated into several mutually exclusive subpopulations, or strata. 
The process by which the sample is constrained to include elements fi'om each category is 
called stratified random sampling (Cooper and Emory, 1995: 221). The sample firom 
each stratum, or aircraft type, was drawn such that it was proportionate to the stratum’s 
share of the population. Proportionate stratified sampling is reported to have higher 
statistical efficiency over simple random sampling (Cooper and Emory, 1995:221). 

Three reasons for a researcher to choose stratified random sampling are (1) to 
increase a sample’s statistical efficiency, (2) to provide adequate data analyzing the 
various subpopulations, and (3) to enable different research methods and procedures used 
in different strata (Cooper and Emory, 1995: 221). In this study, proportionate stratified 
sampling was used to achieve the first two goals. 

Initially, sampling plan requirements were based on a confidence interval of 90% 
to determine the sample size of each aircraft stratum. The sample size for each stratum 
was calculated based on the total number of pilot in each population group using the 
following formula (Krejcie and Morgan, 1970: 608): 

_ z^Nx025 _ 

((l-a)2jc(;\r-l))+(22^0.25) 
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where 


n = sample size required 
N = total population size 
a = precision or confidence level desired (.10) 
z = different factor for each confidence level (1.6449) 

Table 3-2 presents the initial sampling plan and the number of surveys to be 
distributed for each aircraft type. Assuming a 56% return rate (the rate of a previous 
research AFIT master’s thesis survey by Star and Welch (1991), which included the same 
sample strata and similar yet more questions), the number of surveys to be mailed was 
calculated as shown in the last column of the table. Table 3-3 presents the actual number 
of surveys mailed, and the final numbers of surveys responses received 

The ATLAS database of AFPC located on a mainfimne computer at Kelly AFB, 
San Antonio, TX was used to retrieve samples from each stratum based on pilots’ Air 
Force Specialty Codes (AFSC) by HQ ACC/DPO and AMC/DPI. Table 3-4 presents the 
list of specialty codes used in sample generation process by Major Commands’ Persoimel 
Offices. In addition to AFSC, the selection criteria included the pilots assigned to ACC 
and AMC with combat flying hours greater than zero. 
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Table 3-2. Initial Sampling Plan and Survey Data 


Number of Surveys 


Aircraft 

Reauired Sample Size 

to be mailed 

A/AO-10 

56 Pilots 

100 Surveys 

B-52 

51 

90 

C-130 

62 

112 

F-15 

60 

108 

F-16 

63 

113 

KC-135 

63 

113 

Totals: 

352 Pilots 

632 Surveys 


Table 3-3. Actual Survey Data 


Aircraft 

Number of Surveys 
Mailed 

Number of Surveys 
Received 

A/AO-10 

112 Surveys 

34 Surveys 

B-52 

120 

41 

C-130 

133 

32 

F-15 

88 

31 

F-16 

121 

42 

KC-135 

120 

39 

Totals: 

694 Pilots 

219 Surveys 


Table 3-4. AFSC Data 

Aircraft 

AFSC 

A/AO-10 

1115N 

B-52 

1235C 

C-130 

1055B 

F-15 

1115B 

F-16 

1115Q 

KC-135 

1065C 
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A survey package, consisting of a cover letter, a survey questionnaire, an AFIT 
Form lie, and apre-addressed return envelope, was sent to each individual allowing 
adequate time to respond. The respondents were instructed to return the survey 
questionnaire and AFIT Form 11C in the pre-addressed enveloped. 


Statistical Analysis of the Survey Data 

The six investigative questions introduced in the first chapter, questions 2, 
3a through 3d, and 4, were examined by analyzing the data gathered on SAS statistical 
analysis package. Responses to these six questions were analyzed to determine critical 
combat workload items in measuring workload in a combat environment, and a possible 
consensus of the relative priorities of combat workload items among USAF pilots. Data 
provided by answers to these particular questions should help draw conclusions about the 
overall research question: Is it possible to assess pilot workload in a high threat combat 
environment? 

Survey questions 9 through 37 were intended to provide an answer to 
investigative question two, which asked, “Is it possible to determine, by surveying 
experienced pilots, what items would be most important in measuring workload in a high- 
threat environment?” The responses to each combat workload are summed and divided 
by the number of surveys returned to estimate an average numerical value to determine, 
overall, which combat workload items are found important by USAF pilots. The 
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following scale is used to determine the importance of overall ratings for combat 
workload items in assessing workload in high-threat environment. 

If the average value for a particular combat workload item is: 

1.599 or less, it is considered to be a dangerously important combat workload 

item; 

1.600 - 2.599, it is considered as a distractingly important combat workload item; 

2.600 - 3.599, it is considered as a moderately important combat workload item; 

3.600 - 4.599, it is considered as a somewhat important combat workload item; 

4.600 - 5.599, it is considered as a little or not important combat workload item; 

5.600 or higher, it is considered as a not applicable combat workload item. 

The combat workload items are presented in separate tables in Chapter IV 

depending on their importance category. 

Survey questions 9 through 37were also intended to provide an answer to 
investigative question two, which asked, “What do pilots of different aircraft type believe 
are the critical items that would increase their workload in a high threat combat 
environment?” The responses to each combat workload item are grouped by the aircraft 
type. Then average values for each combat workload item are calculated by dividing the 
summed values of a particular combat workload item by the number of the respondents 
from that particular aircraft type. The scale described above is used to draw conclusions 
about the importance of combat workload items for each aircraft in assessing the pilot 
workload in a high-threat environment. The listings of combat workload items based on 
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their importance in workload assessment grouped by the aircraft type are presented in 
separate tables in Appendix D. 

A similar methodology is used to evaluate the responses to investigative question 
3b, which asked, “Does the level of workload perceived by pilots change depending on 
the type of the mission flown?” In this case, however, grouping of combat workload 
items is based on the type of the mission flown. The listings of combat workload items 
based on their importance in workload assessment grouped by the mission type are 
presented in separate tables in Appendix D. 

Responses to investigative question 3c, which asked, “Does the level of workload 
perceived by pilots of each type vary by the experience level of the pilots?” were 
computed in similar fashion. Disregarding the aircraft type, average combat workload 
items are grouped separately based on total flying hours, combat flying hours, and 
previous aircraft experiences of all pilots, excluding trainers. Rank is not considered as 
an experience criterion from the standpoint of flying experience. Occasionally, a captain 
mi ght have more flight experience than a lieutenant colonel or a colonel. B-52, C-130, 
and KC-135 pilots with more than 200 hours of combat and 2,000 hours of total flying 
time, and with at least one previous aircraft qualification except training aircraft are 
considered as “experienced” pilots. Similarly, AO/A-10, F-15, and F-16 pilots with more 
than 100 hours of combat and 1,000 hours of total flying time, and at least one previous 
aircraft qualification other than training aircraft are considered as “experienced” pilots. 
Otherwise, pilots not meeting these criteria are considered as “inexperienced.” Combat 
workload items based on responses by both “experienced” and “inexperienced” pilots. 
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along with average rating and standard deviation values, are listed in the same table in 
Chapter IV. 

Analysis of responses to investigative question 3d, which asked, “Is there 
a particular combat workload item that significantly possesses higher perception of 
workload by all pilots?” is based on the average value of each combat workload item. 
Mean values, calculated during the analysis of the data for the investigative question two, 
were compared among all combat workload items in terms of their lower and upper 95 
percent confidence intervals. It was hypothesized that Threat Avoidance would be valued 
as the most important workload item. For this hypothesis to be true, there would be no 
other workload item whose upper and lower intervals coincided with those of Threat 
Avoidance. 

Finally, to draw conclusions about investigative question four, which asked, “Is it 
possible to enumerate combat workload perceptions of a pilots overall, or those of pilots 
from different aircraft, depending upon their characteristic data?” a variable called CWL 
(Combat Workload) was used. CWL is a variable which measures the overall perception 
of in-flight workload on a combat mission. Overall CWL score for each individual is 
calculated as a summation of responses to significant combat workload items. 

To determine significant combat workload items, Cronbach’s alpha and a 
Principal Component analysis are performed on the twenty-nine combat workload items 
introduced through Questions 9 through 37. For these analyses the SAS statistical 
analysis software package is used. Cronbach’s alpha is a correlation analysis that 
measures the internal consistency from one set of measures to another set of measures 
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(SAS Institute Inc., 1990:213). Cronbach’s alpha is used to secure the reliability 
estimates of the test items, and it has the most utility for multi-item scales at the interval 
level of measurement (Cooper and Emory,1995:155). 

Principal Component Analysis is the most frequently approach used in item 
analysis. This method transforms a set of variables into a new set of variables or 
principal components that are not correlated with each other. These linear combinations 
of variables, called items, accormt for the variance in the data (Cooper and Emory, 1995: 
538). By performing the Cronbach’s alpha and the Principal Component Analysis, the 
appropriateness of the selection of combat workload items to make up the overall CWL 
score of individuals is assured. These calculations should also ensure that the CWL score 
measures the same item for each individual. In addition to individual CWL scores, 
overall average CWL scores for aircraft are determined for each aircraft. After 
performing one-way ANOVA procedure on significant independent variables such as the 
current aircraft flown, the mean CWL score for each aircraft is compared to the overall 
mean scores of each of the other aircraft at 95 percent confidence level using the 
Bonferroni procedure. The Bonferroni procedure compares the mean of different groups 
by using the t-distribution. By comparing means between aircraft group, conclusions 
about possible differences in perceptions of combat workload items between pilots 
groups can be reached. 

By using the data gathered in personal characteristics section of the questionnaire 
as independent variables, a model is developed by stepwise regression routine. This 
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model is expected to predict the pilots’ perception of combat workload items. The 
following questions are used as independent experience variables. 

1. What is your current rank? 

3. How many hours of flight experience do you have, excluding flight training ? 

4. What is the current aircraft you are flying or the most recent aircraft you flew? 

5. What other aircraft have you flown? 

6. How many hours of flight experience do you have in combat? 

8. What primary mission did you fly in combat? 

48. Is the individual an experienced pilot? (The answer was determined based 
on the criteria explained previously.) 

A similar attempt is made to develop for each aircraft a different model 
that can predict the combat workload item perceptions of pilots of that aircraft. The result 
of stepwise routine is a model containing only those independent variables with t values 
that are significant at the pre-determined alpha level. The alpha level for this research 
effort was set to .05, which provides a 95 percent confidence level. 


Statistical Tests 

The SAS statistical analysis package. Version 6.12, was used to performed the 
required analyses. The analyses performed consisted of determining frequency 
responses, computing means, computing the results of one-way Analysis of Variance and 
Bonferroni t-tests, computing result of paired t-tests, comparing the lower and upper 95 
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percent confidence levels of mean values, performing reliability analysis, and performing 
stepwise regression routine on demographics data. 


Summary 

In this chapter, the construction process of the survey and the pre-test 
procedure have been discussed. In addition, population and the sample size of each 
aircraft stratum were described. Furthermore, the chapter discussed the statistical 
analysis to conclude the answers to the investigative questions of the first chapter. 
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IV. Data Analysis 


Chapter Overview 

This chapter presents the data gathered from the respondents of the survey 
questionnaire (included in Appendix A). There are three sections to this chapter: personal 
characteristics, analysis of workload items, and analysis of Combat Workload (CWL) 
scores. Each section describes the data received from US Air Force pilots who 
participated in this survey. These sections also includes the discussion of unexpected or 
unusual facts about the data. 


Personal Characteristics 

Initially, it was intended to survey a sample size large enough to assure a 90% 
.confidence level. Nevertheless, the number of surveys received did not meet the 
requirement for a 90% confidence level on any of the aircraft types. Of 694 surveys 
distributed, only 219 were returned, for a return rate of 31.6%. Figure 4-1 illustrates the 
responses received from individuals grouped by the aircraft type. 

The low return rate could be explained as follows: (1) no preliminary notification 
was made to individuals about the survey package they would receive; (2) no follow-up 
letters were sent to individuals to remind them of the importance of their participation in 
the research effort; and (3) no return postage was furnished on the pre-addressed return 
envelopes. These precautionary methods of improving the survey returns were not able 
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to be accomplished due to time and budgetary constraints. Cooper and Emory (1995) 
note that follow-ups, preliminary notification, and concurrent techniques such as 
questionnaire length, survey sponsorship, return envelopes, postage, personalization, 
cover letters, anonymity, size and color, money incentives, and deadline dates are the 
three important variables for improving the return rate of survey responses (Cooper and 
Emory, 1995:283). 
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Figure 4-1. Survey Returns 

Thirty-six of the survey packages sent were returned to sender because the 
addressees out-processed to a different unit. One of the individuals who returned the 
package was not a pilot. Six of the pilots who responded were flying different aircraft 


4-2 







from those which were included in the survey. These responses were not included in the 
data set. From the figures mentioned in this paragraph, it might be quite possible that 
ATLAS database used in sample generation did not contain the latest records about 
USAF pilots flying the types of aircraft researched. 

Figure 4-2 illustrates the percentages of the surveys responses grouped by aircraft 
types. The figure presents the respondents’ share in the data set used in the statistical 
analysis. Figure 4-3 presents the current military rank distribution of the respondents. 



Figure 4-2. Percentages of Returned Surveys for Each Aircraft 


Almost 50 percent of all respondents held the military rank of captain. Pilots with 
the military rank of major yielded 36% of all respondents. Pilots with the military ranks 





























of captain and major constituted the majority of the data. As T* or 2"“* lieutenants or 
captains of recent major conflicts such as Operation Desert Storm or Operation Deny 
Flight, the majority of the survey sample participated in those conflicts. 



Figure 4-3. Current Rank Distribution of Pilots 


None of the respondents was female, even though survey packages were sent to 
several female pilots. Female pilots first attended flight school in 1977. They had not 
been accepted for training in fighter aircraft until July of 1993. According to 
demographic data released by HQ AFPC, as of 28 April 1997, 342 female pilots serve in 
the United States Air Force, or 2.32 percent of the pilot force. For this reason, gender 
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related differences about the pilots’ perception of workload assessment could not be 
analyzed. 

The remainder of the data on personal characteristics reveals information about 
the experience level and the recent aircraft qualifications of the pilots. The experience 
and qualification areas measured were: (1) the total amount of flight hours accumulated 
on a USAF aircraft other than training models; (2) the total amount of flight hours 
accumulated in combat; and (3) the number of the recent military aircraft types that 
respondents were qualified to fly. The personal characteristic data contain information 
about the types of missions which pilots had flown recently. 

Figures 4-4 and 4-5 depict the total amoimt of flying hours in the six aircraft types 
researched excluding UPT flying hours. A scale from less than 500 hours to more than 
3,000 hours with 500-hour bin width was used. However, the total amount of flying hour 
data from six different aircraft shows that the scale arrangement with 500-hour bin width 
was inappropriate for B-52, C-130, and KC-135 aircraft. The total amount of flying 
hours data for these aircraft, illustrated in Figure 4-5, is skewed to the right. In these 
aircraft, unlike AO/A-10, F-15 and F-16 aircraft, greater flying time is accumulated in 
one sortie. Longer duration missions in these aircraft types allow the pilots to accumulate 
more flying hours in a shorter period of time. In classifying the total amount of flying 
hours, two different aircraft groups are used, “heavy” and “light.” “Heavy” aircraft 
consist of B-52, C-130, and KC-135 aircraft while “light” aircraft include AO/A-10, F-15 
and F-16 aircraft. From the same perspective, the “experienced” and “inexperienced” 
criteria were established for pilots from each aircraft group. “Light” pilots with less than 
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1000 hours of total flying time were considered “inexperienced”; similarly, “heavy” 
pilots with less then 2,000 hours of total flying time were considered “inexperienced.” A 
one to two ratio between inexperience and experience criteria in terms of flying hours 
were determined according the ratio of average duration of sorties between each aircraft 
group. Based on the classification criteria above, approximately 93 percent of “light” 
pilots were “experienced” while 98 percent of “heavy” pilots were experienced. 

Similarly, Figures 4-6 and 4-7 the illustrate total amount of pilot flying hours 
accumulated in combat. To collect the total amount of combat flying hours accumulated 
in each type of the aircraft, a smaller scale was used. The scale included a “none” option, 
and ranged firom equal or less than 100 hours to more than 500 hours with bin width of 
100 hours. 


> 3,000 


2,501-3,000 

2 

3 2,001-2,500 
o 

X 

w 1,501-2,000 
il 

■§ 1001-1,500 


501-1,000 

<500 


0 5 10 15 20 

Number of Pilots 


DAO/A-10 BF-15 0F-16 



Figure 4-4. AO/A-10, F-15, and F-16 Pilots’ Total Flying Hotirs 
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Figure 4-5. B-52, C-130, and KC-135 Pilots’ Total Flying Hours 


Combat experience is an important measure for the piupose of the study. As 
mentioned in the second chapter, this study surveyed USAF pilots’ perception of pilot 
workload assessment in combat or in a high-threat environment. Sample generation 
focused on subjects who had logged at least one combat flying hour. “Inexperienced” 
criteria limits for “light” and “heavy” were established as equal or less than 100 hours 
and 200 hours, respectively. According to these criteria, approximately 54 percent of 
“light” pilots and 11 percent of “heavy” pilots were “experienced” in combat flying hours 
category. It should be noted that percentages of “experienced” pilots of each general 
aircraft classification are similar when total amoimts of flying hours are compared; 
however, those differ considerably when comparing total amoimts of combat flying 
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hours. For both classifications of aircraft, the total amount of combat flying hours data is 
skewed to the left. When Figures 4-6 and 4-7 are examined, combat flight hours 
distributions of “light” and “heavy” pilots are similar. It might appear that the 1:2 ratio 
between combat flying hours of “light” and “heavy” pilots is not reflected in Figures 4-6 
and 4-7. This apparent lack of difference could be due to fact that “light” pilots are 
scheduled for combat missions more frequently in comparison with the “heavy” pilots in 
the same regional conflict. Frequently scheduled combat missions could allow “light” 
pilots to accumulate similar amounts of combat flying hoius in a shorter time compared 
to “heavy” pilots. 



OAO/A-10 BF-15 0F-16 


Figure 4-6. AO/A-10, F-15, and F-16 Pilots’ Combat Flying Hours 


4-8 


















Figure 4-7. B-52, C-130, and KC-135 Pilots’ Combat Flying Hours 


Figures 4-8 and 4-9 illustrate the previous aircraft experiences of pilots of each 
type. The previous aircraft experience data were also presented separately for “light” and 
“heavy” pilots. A clear presentation is achieved when aircraft are divided into two 
groups. 

Data Question 5 of the survey shows that over 60 percent of all pilots had no 
previous operational aircraft experience, excluding all types of military training aircraft. 
At least one previous operational aircraft was flown by 40 percent of all pilots, while 5 
percent and 1 percent of them had experience in two and three previous operational 
aircraft, respectively. Table 4-1 presents the detailed percentages of previous aircraft 
experience of pilots in each aircraft type. Based on the pertaining data, almost 37 percent 
of all pilots flew at least one operational aircraft previously. 
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When the three criteria for “experience,” those of total flying hours, combat flying 
hours, and previous aircraft experience, are considered simultaneously, “experienced” 
pilots yielded only 13.7% of the sample. Of 30 “experienced” pilots, 25 were “light” 
pilots while the remaining five pilots were flying “heavy” aircraft. 

Finally, Figure 4-10 presents the percentage of the recent combat missions flown 
by all pilots. The data indicates that each combat mission had a relatively equeil share. 
This is mainly because six aircraft types selected for the study perform a wide variety of 
missions, and these aircraft represent a good portion of USAF aircraft inventory. 
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Figure 4-8. AO/A-10, F-15, and F-16 Pilots’ Previous Aircraft Experiences 
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Figiare 4-9. B-52, C-130, and KC-135 Pilots’ Previous Aircraft Experiences 


Table 4-1. Previous Experiences of Pilots by Current Aircraft T\ 


Number of Aircraft Flown Previousl 


Current Aircraft 

None 

1 

2 

3 

At least 1 

AO/A-10 

73.4% 

23.5% 

2.9% 

-% 

26.5% 

B-52 

87.8 

4.9 

4.9 

2.4 

12.2 

C-130 

78.1 

18.8 

3.1 

- 

21.9 

F-15 

35.5 

54.8 

6.5 

3.2 

64.5 

F-16 

50.0 

40.9 

9.1 

- 

50.0 

KC-135 

53.8 

43.6 

- 

2.6 

46.2 

Overall 

63.3% 

30.8% 

4.5% 

1.4% 

36.7% 


Furthermore, almost all aircraft types have different models and versions, 
resulting in several different uses in combat. For instance, AO-10 are flown on rescue 
missions, as well as on Close Air Support (CAS) missions. 
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Figure 4-10. Recently Flown Combat Missions 


Abbreviations: 

Str. Detr.: Strategic Deterrence 

Conv. Hvy. Bmb. Ops.: Conventional Heavy Bombing Operations 

Thtr. Airlift: Theater Airlift 

Tac. Airlift: Tactical Airlift 

A/A Refl: Air to Air Refueling 

Air Supr.: Air Superiority 

CAS: Close Air Support 

Air Intr.: Air Interdiction 

CAP.: Combat Air Patrol 


Despite the large span of missions flown by all aircraft, close investigation of the 
data reveals that certain mission types dominate the aircraft t5^es in which these missions 
are flown. When the data were grouped by aircraft type, the following inferences were 
made. Of 33 AO/A-10 pilots, 25 flew CAS missions in combat. Thirty-seven of 41 B-52 
pilots flew Conventional Heavy Bombing Operations in combat. Nearly half of the 
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C-130 pilots flew Theater Airlift missions while the other half flew Tactical Airlift 
missions in combat or a high-threat environment. Of 31 F-15 pilots surveyed, 24 flew 
Air Interdiction missions. The F-16 is the only aircraft type whose mission type is highly 
diverse. However, half of the pilots of this type flew Air Interdiction missions. Almost 
all KC-135 pilots flew Air to Air Refueling missions in combat. 

Overall, examination of personal characteristics of pilots surveyed reveals that the 
subjects of this study were highly experienced. Demographic analysis indicates that the 
sample of this study represents a good portion of the experienced USAF pilot population. 
The subjects of this study are possibly more experienced than the majority of aviators in 
the U.S. Air Force. Almost all subjects have more than 1000 hours of flying time on a 
tactical aircraft. Furthermore, nearly all pilots accumulated some combat flying time, 
while the majority of them logged over 100 hours of combat flying time. A great portion 
of the subjects possess a military rank of captain or higher. More than one third of all 
subjects were previously qualified in at least one type of aircraft. Last, subjects of this 
study have recently flown various types missions in combat. The high level of 
experience of subjects, both in peacetime and wartime, adds credibility to findings of this 
research as well as to the comments of pilots who participated in this study. 


Analysis of Workload Items 

To answer the major research question, several investigative questions were asked 
in the first chapter of the study. One of the questions asked if it is possible to determine. 
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by surveying experienced pilots, what items would be most important in measuring 
workload in a high-threat environment. As explained in the second chapter, to draw 
conclusions about the particular investigative questions, questions 9 through 37 in the 
survey questioimaire were asked. Each question specified a workload item, which was 
common to all aircraft types. Pilots were directed to evaluate each workload item in 
terms of the degree to which it was likely to increase the pilot workload in a combat 
environment, using a five-point Likert scale. Because all tasks do not apply to all six 
aircraft types surveyed, a “not applicable” choice was included in the choices. 

Responses to each workload item were summed and divided by the number of 
surveys returned to estimate an average numerical value to determine overall which 
workload items were found important by USAF pilots. The following scale was used to 
determine overall workload items to conclude that they were important in assessing 
workload in high-threat environment. 

Analysis of All Combat Workload Items 

First, the ratings for combat workload items from pilots of all aircraft types were 
evaluated together. The listings of workload items based on their importance in workload 
assessment in combat or near combat high threat environment are given in Tables D-1 
through D-3 (in Appendix D). Combat workloads in each category are listed in 
descending order of importance within the particular category. A workload item with a 
lower mean value indicates a higher level of perception. Should two or more combat 
workload items be rated equally on the average, the one with less standard deviation 
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value was listed first. When two or more combat workload items had the exact mean and 
standard deviation values, the one with the smallest range of ratings was listed first. Even 
if the range values happened to be the same, then the workload item that appeared first in 
the survey questionnaire was listed first. 

Overall, no combat workload items are found to be “dangerously” important in 
workload assessment. Although 24 out of 29 combat workload items were rated by at 
least one pilot as likely to increase pilot workload “dangerously” in combat, resulting in 
possible extreme delays in the task currently being performed or unsafe situation or 
mission failure, on the average none of those items is found to be dangerously 
important. Likewise, no combat workload items could be concluded to be of “little” or 
“no” importance in workload assessment, according to the ratings of all pilots. Yet, pilots 
rated several combat workload items as likely to increase pilot workload a little or not at 
all. Furthermore, no combat workload items were found to be not applicable among all 
aircraft. It might be implied that the selection of the listing of twenty-nine combat 
workload items was performed properly to represent common tasks on a combat mission. 
Nevertheless, all 29 workload items were evaluated as “not applicable” by at least one 
pilot who participated in the research. 

Based on the mean value of ratings of all pilots surveyed. Threat Avoidance was 
the combat workload item with the highest level of perceived workload, followed by In¬ 
flight Emergency. These combat workload items were only the two which were 
determined to be likely to increase pilot workload “distractingly” in combat. Eleven 
combat workload items were found to be likely to increase pilot workload “moderately.” 
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The remaining sixteen workload items were believed to cause “some” increase in in-flight 
workload. Among all items, Air Refueling Operations was found to be the least 
important combat workload item from the stand point of in-flight workload assessment. 

Question 37a of the survey questionnaire asked pilots to make any additions to the 
29 combat workload items. Four pilots, who selected to comment on the question, 
specified the usage of night sighting devices and Nuclear-Biological-Chemical (NBC) 
protection gear as other items that increase pilot workload in a high-threat environment. 
Their comments deserve attention and should be considered by future researchers of in¬ 
flight workload assessment. 

When all combat workload items’ ratings of pilots from six aircraft types are 
examined together, it is found that the worst workload scenario exists if an in-flight 
emergency occurs while avoiding an enemy threat. This scenario applies to all types of 
aircraft regardless of the type of missions flown. 

Analysis of Combat Workload Items by Aircraft 

In this area, data were grouped by aircraft type and analyzed in the same manner. 
Table 4-2 presents those combat workload items which pilots rated as likely to 
“distractingly” increase pilot workload in combat, segregated by aircraft type. Tables D- 
4 through D-28 of Appendix D list combat workload items at varying levels of perceived 
pilot workload in combat, based on the average ratings of pilots from each aircraft type. 

As reported earlier in Chapter 3, the study intended to identify some in-flight tasks that 
might increase pilot workload “dangerously.” As with overall ratings, surprisingly, no 
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combat workload items were evaluated as “dangerously” important when the pertaining 
data were analyzed within each aircraft type. Therefore, evaluation of “dangerously” 
im portant workload items in each aircraft type was not possible. To save time and effort 
in analysis, examination of combat workload items found to increase pilot workload 
“moderately,” “somewhat,” or “a little” or “not,” and those concluded to be “not 
applicable” for the particular aircraft type are not listed in tables in this section. Instead, 
they are reported only in the text. 

AO/A-10 . Of 29 combat workload items, 23 items were rated as likely to increase 
pilot workload “dangerously” by at least one of 34 AO/A-10 pilots surveyed; however, no 
combat workload item was concluded to be “dangerously” important in pilot workload 
assessment by all AO/A-10 pilots. Five combat workload items found to be 
“distractingly” important among AO/A-10 pilots in workload assessment are listed in 
Table 4-2. Among these five. Threat Avoidance was rated as the most important combat 
workload item. AO/A-10 pilots rated 14 combat workload items, which are listed in 
Table D-5, as “moderately” important items in workload assessment. “Moderately” 
important items consisted of Night Operations, Night Low Level Navigation, Equipment 
Degradation, Fatigue, Target Acquisition, Threat Acquisition, Threat Detection, 
Unfamiliar Terrain, Terrain Avoidance or Terrain Following, Crew Incapacitation, Low 
Level Operation, Shifting Attention to Targets of Opportunity, Command and Control, 
and Management of TOT. Based on the average values of ratings, the following nine 
workload items were found to be “somewhat” important (Table D-6). 
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Table 4-2. Distractingly Important Combat Workload Items by Aircraft Type 


Aircraft 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

AO/A-10 

1. Threat Avoidance 

1.9118 

0.8658 

1.0000 

3.0000 


2. In-flight Emergency 

2.0588 

0.6938 

1.0000 

4.0000 


3. Adverse Weather 

2.2941 

0.9701 

1.0000 

5.0000 


4. Aircraft Maneuvering 

2.3529 

1.5116 

1.0000 

5.0000 


5. In-flight No-notice 
Mission Changes 

2.5882 

0.7437 

1.0000 

4.0000 

B-52 

1. Crew Incapacitation 

1.7692 

1.0378 

1.0000 

5.0000 


2. Threat Avoidance 

1.8974 

0.5522 

1.0000 

3.0000 


3. Fatigue 

2.0513 

0.9163 

1.0000 

4.0000 


4. Night Low Level 
Navigation 

2.3077 

1.0552 

1.0000 

6.0000 


5. In-flight Emergency 

2.4103 

0.7152 

1.0000 

4.0000 


6. Aircraft Maneuvering 

2.4103 

1.0442 

1.0000 

4.0000 

C-130 

1. Threat Avoidance 

2.2500 

0.8032 

1.0000 

4.0000 


2. In-flight Emergency 

2.5000 

1.1359 

1.0000 

5.0000 


3. Adverse Weather 

2.5625 

1.0453 

1.0000 

5.0000 


4. Crew Incapacitation 

2.5625 

1.4797 

1.0000 

6.0000 


5. Fatigue 

2.5938 

0.9456 

1.0000 

4.0000 

F-15 

1. In-flight Emergency 

2.1613 

1.0984 

1.0000 

6.0000 


2. Crew Incapacitation 

2.2258 

1.9098 

1.0000 

6.0000 


3. Threat Avoidance 

2.2903 

0.7829 

1.0000 

4.0000 

F-16 

1. In-flight Emergency 

2.2500 

0.9675 

1.0000 

5.0000 


2. Threat Avoidance 

2.3636 

1.2025 

1.0000 

6.0000 

KC-135 

1. Crew Incapacitation 

2.3590 

1.5129 

1.0000 

6.0000 


2. In-flight Emergency 

2.4359 

0.9402 

1.0000 

5.0000 


3. Threat Avoidance 

2.5385 

1.0475 

1.0000 

5.0000 
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They are, in descending order of importance: Managing Radio Communication, 
Responding to Ground/Air Controllers Instructions, Munitions Deployment, Visual 
Orientation, Formation Responsibilities, Mission Planning, Type of Drop, Monitoring 
Flight Instruments, and Refueling Operations. On the average, AO/A-10 pilots rated 
Crew Coordination as the least important combat workload item, as “a little” or “not” 
important, due to fact that they fly mostly single-seat models (Table D-7). No combat 
workload items were evaluated as “not applicable” on the average, although six out of 29 
workload items are rated as “not applicable” by one or more AO/A-10 pilots. 

B-52 . Though 17 combat workload items were rated as likely to increase pilot 
workload “dangerously” by one or more B-52 pilots surveyed, no combat workload item 
was concluded to be “dangerously” important in pilot workload assessment. Six combat 
workload items were found to be “distractingly” important by 40 B-52 pilots, and they 
are listed in Table 4-2. Among all items. Crew Incapacitation was rated as the highest 
important combat workload item. B-52 pilots’ average ratings identified 14 combat 
workload items presented in Table D-9 as “moderately” important items in combat 
workload assessment. The fourteen “moderately” combat workload items, in descending 
order of importance, were In-flight No-notice Mission Changes, Adverse Weather, 
Terrain Avoidance or Terrain Following, Night Operations, Equipment Degradation, 
Unfamiliar Terrain, Low Level Navigation, Maintaining Situational Awareness, Threat 
Detection, Crew Coordination, Target Acquisition, Managing Radio Communication, 
Command and Control, and Formation Responsibilities. Among these workload items. 
Target Acquisition and Managing Radio Communication demonstrated comparable 
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ratings as important combat workload items. According to the average rating values, nine 
B-52 “somewhat” important combat workload items are, in the descending order of 
importance: Responding to Ground or Air Controllers Instructions, Management of TOT, 
Visual Orientation, Munitions Deployment, Type of Drop, Shifting Attention to Targets 
of Opportimity, Monitoring Flight Instruments, Refueling Operations, and Mission 
Planning (Table D-10). On the average, B-52 pilots chose Mission Planning as the least 
important combat workload item. No combat workload item was found to be “a little” or 
“not” important item, and there were no “not applicable” items. However, each of 10 
different combat workload items was rated as “not applicable” by at least one B-52 pilot. 

C-130 . Despite the fact that 21 combat workload items were evaluated as likely 
to increase pilot workload “dangerously” by a minimum of one C-130 pilot, no combat 
workload item was considered as “dangerously” important overall in pilot workload 
assessment. Based on the evaluation of 32 C-130 pilots, five combat workload items 
were identified as “distractingly” important, which are presented in Table 4-2. Threat 
Avoidance was evaluated as the most important combat workload item. The following 
items, also listed in Table D-12, are fifteen combat workload items evaluated as 
“moderately” important items in workload assessment according to C-130 pilots’ average 
ratings: Aircraft Maneuvering, In-flight No-notice Mission Changes, Command and 
Control, Terrain Avoidance or Terrain Following, Maintaining Situational Awareness, 
Night Low Level Navigation, Managing Radio Communication, Threat Detection, 
Equipment Degradation, Low Level Navigation, Unfamiliar Terrain, Formation 
Responsibilities, Night Operations, Mission Planning, and Crew Coordination. C-130 
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pilots’ average ratings specified Visual Orientation, Target Acquisition, Management of 
TOT, Responding to Ground Airborne Controller Instructions, Monitoring Flight 
Instruments and Type of Drop as the six “somewhat” important combat workload items 
(Table D-13). Shifting the Attention to Targets of Opportunity was valued as of “a little” 
or “no” importance (Table D-14). The measuring scale used in the survey was successful 
in detecting Munitions Deployment and Refueling Operations as “not applicable” to C- 
130 combat operations, because most C-130 aircraft are not equipped with munitions 
deployment or air-to-air refueling systems (Table D-15). 

F-15 . Evaluations of 31 F-15 pilots revealed that no combat workload item was 
considered as “dangerously” important. On the other hand, thirteen combat workload 
items were rated as likely to increase pilot workload “dangerously” by one or more F-15 
pilots. Three workload items, presented in Table 4-2, were measured as “distractingly” 
important. Their highest perceived combat workload item choice was In-flight 
Emergency followed by Crew Incapacitation. Threat Avoidance was rated as the third 
most important combat workload item. F-15 pilots’ average ratings determined nine 
combat workload items presented in Table D-17 as “moderately” important items in 
combat workload assessment. They were Aircraft Maneuvering, Fatigue, Equipment 
Degradation, Adverse Weather, In-flight No-notice Mission Changes, Terrain Avoidance 
or Terrain Following, Shifting Attention to Targets of Opportunity, Maintaining 
Situational Awareness, and Threat Detection. According to the average rating values, the 
remaining 15 combat items were found to be “somewhat” important when assessing 
workload. They are, in descending order of importance. Target Acquisition, Night 
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Operations, Low Level Navigation, Night Low Level Navigation, Command and Control, 
Unfamiliar Terrain, Munitions Deplo 5 mient, Visual Orientation, Formation 
Responsibilities, Mission Plaiming, Managing Radio Communication, Management of 
TOT, Type of Drop, Responding to Ground or Airborne Controller Instructions, 
Monitoring Flight Instruments, Crew Coordination, and Air Refueling Operations (Table 
4-18). On the average, F-15 pilots’ choice of the least important combat workload item 
was Air Refueling Operations. No combat workload item was found to be “a little” or 
“not” important item at all. Although almost two thirds of the combat workload items 
was rated as “not applicable” by at least one pilot, none was valued as “not applicable” on 
the average. 

F-16 . Although 13 combat workload items were rated as likely to increase pilot 
workload “dangerously” by one or more F-16 pilots surveyed, no combat workload item 
was concluded to be “dangerously” important in pilot workload assessment on the 
average. Only In-flight Emergency and Threat Avoidance were valued as “distractingly” 
important by 44 F-16 pilots, the former rated as the most important. The two 
“distractingly” important workload items are presented in Table 4-2 along with the mean 
value scores. Upon examination of F-16 pilots’ ratings, twelve combat workload items 
were found as “moderately” important items in combat workload assessment. Twelve 
combat workload items are, also listed in Table D-20, Adverse Weather, In-flight No¬ 
notice Mission Changes, Fatigue, Aircraft Maneuvering, Equipment Degradation, Night 
Operations, Threat Detection, Night Low Level Navigation, Target Acquisition, Shifting 
Attention to Targets of Opportunity, Maintaining Situational Awareness, and Crew 


4-22 



Incapacitation. Based on the average rating values, thirteen F-16 “somewhat” important 
combat workload items are, in descending order of importance. Unfamiliar Terrain, Low 
Level Navigation, Terrain Avoidance or Terrain Following, Mtmitions Deployment, 
Managing Radio Commimications, Command and Control, Responding to Ground or 
Airborne Radar Controller Instructions, Management of TOT, Mission Planning, 
Formation Responsibilities, Monitoring Flight Instruments, and Type of Drop (Table D- 
21). Air Refueling was valued as a “little” or “not” important combat workload item on 
the average (Table D-21). Crew Coordination was rated as “not applicable” to F-16 
aircraft due to fact that all F-16s, except training models, are single-seated fighter aircraft. 

KC-135 . Fifteen out of twenty-nine combat workload items were rated as likely 
to increase pilot workload “dangerously” by at least KC-135 pilots surveyed, but no 
combat workload item was concluded to be “dangerously” important in pilot workload 
assessment on the average. Three combat workload items, which are found to be 
“distractingly” important by 39 KC-135 pilots, are listed in Table 4-2. Among all items. 
Crew Incapacitation was rated as the highest important combat workload item. Table D- 
25 presents six combat workload items, that KC-135 pilots valued as “moderately” 
important items in combat workload assessment. They are Fatigue, Adverse Weather, 
Equipment Degradation, In-flight No-notice Mission Changes, Formation 
Responsibilities, and Aircraft Maneuvering. According to the average rating values, 
twelve “somewhat” important combat workload items for KC-135s are, in descending 
order of importance. Command and Control, Maintaining Situational Awareness, 
Managing Radio Communication, Refueling Operations, Threat Detection, Crew 
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Coordination, Night Operations, Visual Orientation, Responding to Ground or Airborne 
Controller Instructions, Mission Planning, Unfamiliar Terrain, and Monitoring Fli ght 
Instruments (Table D-26). On the average. Terrain Avoidance or Terrain Following, 
Night Low Level Navigation, Management of TOT, and Shifting Attention to Targets of 
Opportunity are valued as “a little” or “not” important combat workload items (Table D- 
27). Target Acquisition, Type of Drop, and Munitions Deployment are found “not 
applicable” to KC-135 aircraft (Table D-28). 

When Table 4-2 was reviewed closely, Threat Avoidance and In-flight Emergency 
were rated as likely to increase pilot workload “distractingly” in combat. Threat 
Avoidance was present in all aircraft types within the top three items, and it was rated 
highest in two aircraft types. In-flight Emergency was present in all aircraft listings 
within the top five combat workload items, and it was rated highest in two aircraft types. 
Other common combat workload items found “distractingly” important across aircraft 
types were Crew Incapacitation, Fatigue, Adverse weather, and Aircraft Maneuvering. 
Among these combat workload items. Crew Incapacitation was rated highest in two 
aircraft types. It was interesting to find that Fatigue was not considered as “distractingly” 
important item by fighter pilots; however, it was present in the listing of “distractingly” 
important combat workload items of B-52 and C-130 pilots who fly considerably longer 
missions. AO/A-10, B-52 and C-130 pilots rated more items as “distrac tingl y” important 
than pilots of F-15, F-16, and KC-135 aircraft. Advanced technologies used in the latter 
group of aircraft might be a factor in the lower number of items in this category. 
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Overall, it might be concluded that, regardless of aircraft type, the worst scenario 
of pilot workload in combat exists when an in-flight emergency occurs while avoiding a 
known enemy threat. The same of degree of commonality was not observed when 
“moderately,” “somewhat,” “a little” or “not” important combat workload items were 
examined. 


Analysis of Combat Workload Items by Missions Flown 

In this analysis, data were grouped by missions most previously flown. The 
segregated data was analyzed as in the analysis of the data grouped by aircraft. Tables 
4-3a and 4-3b present the combat workload items which pilots believe “moderately” 
increase combat workload by the most recent combat mission flown. Tables D-29 
through D-64 list combat workload items in varying levels of importance categories 
based on the average ratings of pilots fi'om each mission type. Evaluation of the pilots’ 
ratings of combat workload items who flew Strategic Deterrence missions and “Other” 
missions in combat were neglected, because only two pilots flew Strategic Deterrence 
missions recently in combat, and because fifteen pilots specified that they flew various 
missions other than those provided in the survey questionnaire. Nevertheless, those 
combat workload items having “distractingly” important ratings in Strategic Deterrence 
and “other” missions are listed in Table 4-3b. As reported earlier in Chapter 3, it was 
intended to identify in-flight tasks that were believed to increase pilot workload 
“dangerously.” As in the overall and aircraft groups’ ratings, surprisingly, it was foimd 
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that no combat workload items were evaluated as “dangerously” important when the 
pertaining data were analyzed within each mission type. To save time and effort in 
analysis, examination of combat workload items found to be increase pilot workload 
“moderately,” “some,” “a little” or “not,” and those rated as “not applicable” for the 
particiilar mission type are not listed in tables in this section. Instead, they are reported 
only in the text. The complete listing of items with means and standard deviations can be 
found in Appendix D. 


Table 4-3a. Distractingly Important Combat Workload Items by Mission Type 


Missions 

Flown 

n 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

Conventional 

38 

1. Threat Avoidance 

1.8611 

0.5929 

1.0000 

3.0000 

Heavy 


2. Crew Incapacitation 

1.8611 

1.2684 

1.0000 

6.0000 

Bombing 


3. Fatigue 

2.0556 

0.9545 

1.0000 

4.0000 



4. Night Low Level 

2.2778 

1.0853 

1.0000 

6.0000 



Navigation 

5. In-flight Emergency 

2.3611 

0.6825 

1.0000 

4.0000 



6. Aircraft Maneuvering 

2.4444 

1.0541 

1.0000 

6.0000 

Theater 

13 

1. Threat Avoidance 

2.0000 

0.5774 

1.0000 

3.0000 

Airlift 


2. In-flight Emergency 

2.2308 

1.0919 

1.0000 

5.0000 



3. Crew Incapacitation 

2.3077 

1.2506 

1.0000 

5.0000 



4. Adverse Weather 

2.3846 

0.7679 

1.0000 

4.0000 

Tactical 

18 

1. Threat Avoidance 

2.2778 

0.8264 

1.0000 

3.0000 

Airlift 


2. Fatigue 

2.4444 

0.9218 

1.0000 

4.0000 



3. In-flight Emergency 

2.5556 

1.1991 

1.0000 

5.0000 

Air/Air 

37 

1. Crew Incapacitation 

2.4324 

1.5191 

1.0000 

6.0000 

Refueling 


2. In-flight Emergency 

2.4865 

0.9316 

1.0000 

5.0000 



3. Threat Avoidance 

2.5676 

1.0682 

1.0000 

5.0000 
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Table 4-3b. Distractingly Important Combat Workload Items by Mission Type 


Missions 

Flown 

n 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum Maximum 

Air 

8 

1. Fatigue 

2.0000 

1.0690 

1.0000 

4.0000 

Superiority 


2. Threat Avoidance 

2.2500 

0.7071 

1.0000 

3.0000 



3. In-flight Emergency 

2.3750 

1.1877 

1.0000 

5.0000 



4. Adverse Weather 

2.5000 

1.3093 

1.0000 

4.0000 

Close Air 

26 

1. Threat Avoidance 

1.8462 

0.8339 

1.0000 

3.0000 

Support 


2. In-flight Emergency 

2.1154 

0.7114 

1.0000 

4.0000 



3. Adverse Weather 

2.3462 

1.0561 

1.0000 

5.0000 



4. Aircraft Maneuvering 

2.3846 

1.0612 

1.0000 

4.0000 



5. In-flight No-notice 

2.5385 

0.8115 

1.0000 

4.0000 



Mission Changes 





Air 

54 

1. In-flight Emergency 

2.2593 

1.0494 

1.0000 

6.0000 

Interdiction 


2. Threat Avoidance 

2.4259 

1.0569 

1.0000 

6.0000 

Combat Air 

10 

In-flight Emergency 

2.0000 

0.9428 

1.0000 

4.0000 

Patrol 


Threat Avoidance 

2.3000 

1.0593 

1.0000 

4.0000 

Strategic 

2 

1. Threat Avoidance 

2.0000 

0.0000 

2.0000 

2.0000 

Deterrence 


2. In-flight Emergency 

2.0000 

0.0000 

2.0000 

2.0000 



3. Aircraft Maneuvering 

2.0000 

0.0000 

2.0000 

2.0000 



4. Terrain Avoidance/ 

2.5000 

0.7071 

2.0000 

3.0000 



Terrain Following 







5. Adverse Weather 

2.5000 

0.7071 

2.0000 

3.0000 



6. In-flight No-notice 

2.5000 

0.7071 

2.0000 

3.0000 



Mission Changes 







7. Fatigue 

2.5000 

0.7071 

2.0000 

3.0000 

Others 

15 

1. Threat Avoidance 

2.1333 

1.0601 

1.0000 

4.0000 



2. In-flight Emergency 

2.2000 

0.8619 

1.0000 

4.0000 



3. Aircraft Maneuvering 

2.3333 

0.8165 

1.0000 

4.0000 



4. Adverse Weather 

2.3333 

0.9759 

1.0000 

4.0000 



5. In-flight No-notice 

2.4667 

0.9155 

1.0000 

4.0000 



Mission Changes 
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Conventional Heavy Bombing Missions . Thirty-six pilots stated that they flev^ 
Conventional Heavy Bombing Missions in combat most recently. All pilots of this group 
flew the specified mission in B-52 aircraft. Combat workload items specified in each 
category of importance were exactly as those reported B-52 combat workload items. 
Based on the mean values of pilots ratings of those who flew Conventional Heavy 
Bombing missions, six combat workload items listed in Table 4-3 a were identified as 
“distractingly” increasing pilot workload in combat. Among all combat workload items. 
Threat Avoidance was valued as the most important item. Fifteen combat workload 
items were evaluated as “moderately” important pilot workload. These are Adverse 
Weather, In-flight No-notice Mission Changes, Terrain Avoidance or Terrain Following, 
Equipment Degradation, Night Operations, Low Level Navigation, Maintaining 
Situational Awareness, Unfamiliar Terrain, Threat Detection, Crew Coordination, Target 
Acquisition, Command and Control, Managing Radio communication. Formation 
Responsibilities, and Responding to Groimd or Airborne Controller Instructions (Table 
D-30). According to the average rating values of pilots of this mission type, nine 
“somewhat” important workload items are, in descending order of importance: 
Responding to Ground/Air Controllers Instructions, Management of TOT, Visual 
Orientation, Munitions Deployment, Type of Drop, Shifting Attention to Targets of 
Opportunity, Monitoring Flight Instruments, Refueling Operations, and Mission Planning 
(Table D-31). On the average, pilots of this mission type chose Mission Planning as the 
least important combat workload item. No combat workload item was found to be as “a 
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little” or “not” important, and there were no “not applicable” items. However, ten 
combat workload items were rated as “not applicable” at least one pilot. 

Theater Airlift Missions . Thirteen pilots reported that they most recently flew 
Theater Airlift Missions in combat. All pilots of this group flew the particular mission in 
C-130 aircraft. Based on the mean values, four combat workload items listed in 
Table 4-3 a were identified as “distractingly” increasing pilot workload in combat. 
Among all combat workload items. Threat Avoidance was valued as the most important 
item. Fourteen combat workload items that were evaluated as “moderately” important 
workload items. They were Fatigue, Aircraft Maneuvering, In-flight No-notice Mission 
Changes, Command and Control, Managing Radio Communication, Maintaining 
Situational Awareness, Equipment Degradation, Night Low Level Navigation, Threat 
Detection, Terrain Avoidance or Terrain Following, Low Level Navigation, Unfamiliar 
Terrain, Formation Responsibilities, and Visual Orientation (Table D-33). Pilots flying 
Theater Airlift missions in combat specified the following nine “somewhat” important 
combat workload items. They are, ranked in descending order of importance; Night 
Operations, Crew Coordination, Target Acquisition Mission Planning, Management of 
TOT, Monitoring Flight Instruments, Responding to Ground or Airborne Controller 
Instructions, and Type of Drop (Table D-34). Shifting Attention to Targets of 
Opportunity was foxmd to be the only “a little” or “not” important item (Table D-35). 
Munitions Deployment and Air-to-Air Refueling Operations were identified as “not 
applicable” items (Table D-36). 
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Tactical Airlift Missions . Eighteen pilots reported that they most recently flew 
Tactical Airlift Missions in combat. Like pilots flying Theater Airlift missions, all pilots 
in this group also flew the mission in C-130 aircraft. According to average values of 
pilots’ ratings of this mission type, three combat workload items listed in Table 4-3a were 
identified as “distractingly” increasing important. In this group also. Threat Avoidance 
was rated as the most important item. Table D-38 lists eighteen combat workload items 
that were evaluated as “moderately” important. They were Adverse Weather, Aircraft 
Maneuvering, Crew Incapacitation, Terrain Avoidance and Terrain Following, 
Maintaining Situational Awareness, Command and Control, Night Low Level 
Navigation, In-flight No-notice Mission Changes, Threat Detection, Managing Radio 
Communication, Mission Plaiming, Night Operations, Unfamiliar Terrain, Equipment 
Degradation, Low Level Navigation, Formation Responsibilities, Crew Coordination, and 
Visual Orientation. Among these eighteen items, pilots rated Adverse Weather and 
Aircraft Maneuvering evenly. Pilots who flew Tactical Airlift missions in combat 
specified five “somewhat” important combat workload items. They were Target 
Acquisition, Responding to Ground or Airborne Controllers Instructions, Management of 
TOT, Type of Drop, and Monitoring Flight Instruments (Table D-39). Shifting Attention 
to Targets of Opportunity was specified as “a little” or “not” important combat workload 
item (Table D-40). Like the pilots flying Theater Airlift mission, the pilots of this 
mission type rated Munitions Deployment and Air-to-Air Refueling Operations as “not 
applicable” items. 
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Air-to-Air Refueling Missions . Thirty-seven pilots who most recently flew Air- 
to-Air Refueling missions were KC-135 pilots. For this reason, combat workload items 
specified in each category of importance were exactly same as those of KC-135 pilots. 
Pilots of this mission type rated three combat workload items as “distractingly” 
increasing pilot workload in combat. They are listed in Table IV-3a. In this mission 
type, the combat workload item with highest level of importance was Crew 
Incapacitation. Table D-43 lists six combat workload items that were valued as 
“moderately” important. They were Fatigue, Adverse Weather, Equipment Degradation, 
In-flight No-notice Mission Changes, Formation Responsibilities, and Aircraft 
Maneuvering. Maintaining Situational Awareness, Command and Control, Managing 
Radio Communication, Threat Detection, Crew Coordination, Night Operations, Visual 
Operations, Responding to Groimd or Airborne Controller Instructions, Refueling 
Operations, Mission Planning, Unfamiliar Terrain, and Monitoring Flight Instrument 
were identified as “somewhat” important combat workload items in combat (Table D-44). 
The following five combat workload items were considered as “a little” or “not” 
important workload items. These are Terrain Avoidance or Terrain Following, Night 
Low Level Navigation, and Shifting Attention to Targets of Opportunity (Table D-45). 
Target Acquisition, Type of Drop and Munitions Deployment were foimd to be “not 
applicable” to the particular mission type (Table D-46). 

Air Superiority Missions . Six F-15 pilots and two F-16 pilots stated that they 
most recently flew Air Superiority missions in combat. Pilots of this mission type rated 
four combat workload items presented in Table IV-3b as “distractingly” important. In 
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this mission type, fatigue was rated as the most important combat workload item. Table 
D-48 presents seven combat workload items that were identified as “moderately” 
important in combat. They are Terrain Avoidance or Terrain Following, Crew 
Incapacitation, Night Operations, In-flight No-notice Mission Changes, Aircraft 
Maneuvering, Threat Detection, and Low Level Navigation. The following thirteen 
combat workload items were evaluated as “somewhat” important: Equipment 
Degradation, Shifting Attention to Targets of Opportimity, Maintaining Situational 
Awareness, Command and Control, Unfamiliar Terrain, Managing Radio 
Communications, Munitions Deployment, Formation Responsibilities, Mission Planning, 
Target Acquisition, Monitoring Flight Instruments, Night Low Level Navigation, and 
Visual Orientation (Table D-49). Management of TOT, Responding to Ground or 
Airborne Controller Instructions, Refueling Operations, and Type of Drop were four 
combat workload items considered as of “little” or “no” importance (Table D-50). Crew 
Coordination was found to be “not applicable” to the particular mission type (Table D- 
51). 

Close Air Support . Twenty-five of 26 pilots who most recently flew Close Air 
Support missions in combat were AO/A-10 pilots, along with one F-16 pilot. Pilots 
flying this mission type rated four combat workload items presented in Table IV-3b as 
“distractingly” important in combat. Among these four. Threat Avoidance was the most 
important combat workload item. Thirteen combat workload items were identified as 
“moderately” important. They are Fatigue, Night Low Level Navigation, Night 
Operations, Equipment Degradation, Threat Detection, Target Acquisition, Terrain 
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Avoidance or Terrain FolloAving, Crew Incapacitation, Unfamiliar Terrain, Maintaining 
Situational Awareness, Low Level Navigation, Shifting Attention to Targets of 
Opportunity, and Management of TOT (Table D-53). Ten combat workload items were 
found to be “somewhat” important workload items. They are Command and Control, 
Munitions Deployment, Managing Radio Communication, Responding to Ground or 
Airborne Controller Instructions, Visual Orientation, Formation Responsibilities, Mission 
Planning, Type of Drop, Monitoring Flight Instruments, and Refueling Operations (Table 
D-54). Crew Coordination was foimd to be “a little” or “not” important workload item 
(Table D-55). No combat workload item was rated as “not applicable” to this mission 
type. 

Air Interdiction . Among the 54 pilots who most recently flew Air Interdiction 
missions in combat were six AO/A-10, two B-52, twenty-four F-15, and twenty-two F-16 
pilots. Pilots’ ratings of this mission type identified two combat workload items as 
“distractingly” important in combat. In-flight Emergency was rated as the most 
important one of all 29 combat workload items. Table 4-3b lists two “distractingly” 
important combat workload items. Twelve combat workload items with “moderately” 
important rating are listed in Table D-57. They were Aircraft Maneuvering, Adverse 
Weather, Equipment Degradation, In-flight No-notice Mission Changes, Crew 
Incapacitation, Fatigue, Night Low Level Navigation, Target Acquisition, Shifting 
Attention to Targets of Opportunity, Threat Detection, Maintaining Situational 
Awareness, and Night Operations. Terrain Avoidance or Terrain Following, Unfamiliar 
Terrain, Low Level Navigation, Munitions Deployment, Command and Control, Visual 
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Orientation, Managing Radio Communications, Management of TOT, Responding to 
Ground or Airborne Controllers Instructions, Formation Responsibilities, Mission 
Planning, Type of Drop, Monitoring Flight Instruments, and Refueling were the fifteen 
combat workload items with “somewhat” important workload rating (Table D-58). Crew 
Coordination was found to be of “little” or “no” importance (Table D-59). No combat 
workload item was valued as “not applicable” to this mission type. 

Combat Air Patrol . Ten pilots reported that they most recently flew Combat Air 
Patrol (CAP) missions in combat. Among those ten pilots were one AO/A-10, one F-15, 
and eight F-16 pilots. Pilots flying this mission type rated two combat workload items as 
“distractingly” increasing pilot workload in combat. In-flight Emergency was rated as 
the most important one of all combat workload items. Table 4-3b lists two 
“distractingly” important combat workload items. Table D-61 lists nine combat 
workload items vrith “moderately” important rating. They are In-flight No-notice 
Mission Changes, Adverse Weather, Equipment Degradation, Fatigue, Threat Detection, 
Aircraft Maneuvering, Shifting Attention to Targets of Opportimity, Crew Incapacitation, 
and Unfamiliar Terrain. Pilots of this mission type determined the follovring fifteen 
combat workload items as having “somewhat” important rating. They are, listed in 
descending order of importance. Command and Control, Night Operations, Visual 
Orientation, Maintaining Situational Awareness, Target Acquisition, Night Low Level 
Navigation, Low Level Navigation, Mission Planning, Managing Radio 
Communications, Refueling Operations, Munitions Deployment, Type of Drop, 

Formation Responsibilities, Responding to Groimd or Airborne Controllers Instructions, 
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and Terrain Avoidance and Terrain Following (Table D-62). Management of TOT and 
Monitoring Flight Instruments were rated as of “little” or “no” importance (Table D-63). 
Crew Coordination was evaluated as “not applicable” to this mission type (Table D-64). 

A close examination of Tables 4-3a and 4-3b reveals that Threat Avoidance and 
In-flight Emergency were rated “distractingly” important within each mission group. 
Threat Avoidance was rated highest in four out of eight mission group evaluated, and it 
was present within the top three items of all mission types’ listings. In-flight Emergency 
was present within the top five items of all mission groups, where it was rated as the most 
important item in two mission groups. Other common “distractingly” important combat 
workload items were Adverse Weather, Aircraft Maneuvering, Crew Incapacitation, and 
Fatigue, where Crew Incapacitation and Fatigue was present in top listings at least once. 
The combat workload items’ listings of varying importance categories of Conventional 
Heavy Bombing missions and Air-to-Air Refueling missions were exactly same as those 
listings of B-52 and KC-135 aircraft, respectively because those missions were flown in 
the particular aircraft types. However, analysis produced different results for the combat 
workload items’ listings of the missions flown in several different aircraft t 5 ^es than 
those listings of individual aircraft type. For instance, though Fatigue was not present in 
the listing of “distractingly” important combat workload items of F-15 or F-16 aircraft 
types, it was the top item in the listing of Air Superiority missions’ “distractingly” 
important combat workload items. Although Tactical Airlift and Theater Airlift were 
flown on the same type of aircraft, C-130 Hercules, the listings of combat workload items 
varied between these two mission types. On both missions the survey was successful to 
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distinguish the same “not applicable” items for C-130 aircraft. These facts show that the 
survey was able to distinguish the workload items for different types of missions 
provided that the particular mission type are flown on different aircraft, or two missions 
are flown on the same type of aircraft. 

Looking at the Tables 4-3 a and 4-3 b, it could be concluded that regardless of 
mission type flown, the worst scenario of pilot workload in combat exists when an in¬ 
flight emergency occurs while avoiding an enemy threat. The same degree of 
commonality is not observed in examining “moderately,” “somewhat,” and “a little” or 
“not” important combat workload item listings. 

Analysis of Combat Workload Items Grouped by Experience Level 

To compare the average ratings of “experienced” and “inexperienced” pilots, 
average combat workload items’ ratings were evaluated separately based on the total 
flight hours, combat flight hours, and previous aircraft experiences excluding trainers. 

For B-52, C-130, and KC-135 pilots, “experienced” pilots were those with combat flying 
totaling more than 200 hours, and minimum total flying time of 2,000 hours, and at least 
one previous aircraft qualification. Similarly, for AO/A-10, F-15, and F-16 pilots 
“experienced” pilots had more than 100 combat flying hours, a minimum total flying time 
of 1,000 hours, and at least one previous aircraft qualification. Pilots not meeting these 
criteria were considered as “inexperienced.” After segregating the data based on 
experience criteria, it was foimd that the data set contained 30 experienced and 189 
inexperienced pilots. Tables 4-4a and 4-4b present the list of all combat workload item 
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responses linked to both “experienced” and “inexperienced” pilots, along with average 
rating and standard deviation values. Those “experienced” pilots’ combat workload item 
ratings found to have significantly greater means than those of “inexperienced” pilots 
were marked with triple-asterisks. 

In the first chapter, it was hypothesized that perception of combat workload of 
pilots with more flight experience would likely to be lower than those of inexperienced 
pilots. It should be remembered that, according to the five-point scale provided, the 
higher mean value of an item’s rating implies the lower perception of workload by pilots 
on the particular combat workload item. To test this hypothesis, a two-tailed t-test was 
performed on every pair of combat workload items from “experienced” and 
“inexperienced” pilots’ listings. The two-tailed t-test tested for a 95 percent confidence 
level. For the purpose of the test the null and the alternative hypotheses were modified as 
below. 


Null Hypothesis: The difference between the mean values of “experienced” and 
“inexperienced pilots’ ratings = 0 

Alternative Hypothesis: The difference between the mean values of “experienced” 
and “inexperienced pilots’ ratings > 0 

To be conservative, the results of the test were determined assuming that sample 
pairs did not have equal variance. Based on the results of two-tailed t-test, only 
Managing Radio Communications were rated significantly less among “experienced” 
pilots in comparison with “inexperienced” pilots (Table 4-4b). 
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Table 4-4a. Comparison of Combat Workload Items 


Experienced _ Inexperienced Unequal Variance 

Combat Workload Item Mean Std.Dev. Mean Std.Dev. T-Value P-Value 


1. Mission Planning 

2. Terrain Avoidance/ 
Terrain Following 

3. Maintaining Situational 
Awareness 

4. Adverse Weather 

5. Monitoring Flight 
Instruments 

6. Equipment Degradation 

7. Low Level Navigation 

8. Night Low Level 
Navigation 

9. Threat Avoidance 

10. Formation 
Responsibilities 

11. Management of TOT 

12. In-flight No-notice 
Mission Changes 

13. Shifting Attention to 
Targets of Opporftmity 

14. Mxmitions Deployment 

15. Threat Detection 

16. Crew Incapacitation 

17. In-flight Emergency 

18. Visual Orientation 

19. Command and Control 
(such as copying and 
decoding EAMS) 

20. Fatigue 

21. Crew Coordination 


3.9000 

1.3222 

4.1368 

3.3333 

1.1547 

3.4868 

3.2667 

1.1725 

3.3386 

2.6000 

1.0034 

2.6667 

4.4000 

0.8944 

4.2857 

2.8667 

1.0080 

2.9894 

3.9667 

1.2452 

3.7037 

3.2667 

1.5298 

3.3704 

2.3667 

1.1592 

2.1905 

4.0667 

0.9803 

3.7407 

4.1667 

0.7466 

4.1111 

2.7333 

0.8277 

2.8677 

3.4333 

1.1043 

4.2487 

4.1667 

1.2617 

4.5714 

3.4000 

1.0700 

3.3280 

2.2000 

1.6897 

2.6984 

2.3000 

0.9879 

2.3069 

3.9310 

1.3870 

3.8883 

3.7333 

1.3374 

3.5904 

2.6000 

1.3797 

2.6349 

4.7000 

1.2077 

4.3175 


1.1648 

-0.9300 

0.8197 

1.4052 

-0.6500 

0.7420 

1.1353 

-0.3100 

0.6222 

1.0468 

-0.3400 

0.6307 

0.9009 

0.6500 

0.2599 

0.9839 

-0.6200 

0.7311 

1.2953 

1.0700 

0.1459 

1.6823 

-0.3400 

0.6322 

0.8966 

0.8000 

0.2158 

0.9684 

1.6900 

0.0491 

1.0383 

0.3600 

0.3615 

0.9274 

-0.8100 

0.7893 

1.4464 

-3.5900 

0.9996 

1.2426 

-1.6400 

0.9450 

1.2197 

0.3400 

0.3695 

1.9458 

-1.4700 

0.9253 

0.9289 

-0.0400 

0.5142 

1.1531 

0.1600 

0.4378 

1.1269 

0.5500 

0.2913 

1.0714 

-0.1300 

0.5523 

1.4085 

1.5700 

0.0615 
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Table 4-4b. Comparison of Combat Workload Items 


Experienced Inexperienced Unequal Variance 
Combat Workload Item Mean Std.Dev. Mean Std.Dev. T-Value P-Value 


22. Aircraft Maneuvering 
(Dogfight or 
avoiding the threats) 

23. Target Acquisition 

24. Type of Drop 

25. Night Operations 

26. Unfamiliar Terrain 

27. Managing Radio *’* 
Commxmication 

28. Refueling Operations 

29. Responding to Ground/ 
Airborne Controller 
Instructions. 


2.9333 

1.5071 

2.7513 

3.6333 

1.4016 

3.8466 

4.3448 

1.2034 

4.5397 

3.4333 

1.1651 

3.3333 

3.8667 

1.0743 

3.5661 

4.3333 

0.8841 

3.6667 

4.8000 

0.6103 

4.5661 

4.0665 

1.0419 

3.9781 


1.2576 

0.6300 

0.2671 

1.3925 

-0.7700 

0.7784 

1.2181 

-0.8100 

0.7886 

1.0106 

0.4400 

0.3297 

1.0974 

1.4200 

0.0819 

0.9340 

3.8100 

0.0002 

0.9742 

1.7700 

0.0410 

0.9315 

0.4400 

0.3301 


Confidence Interval Analysis of Combat Workload Items: Threat Avoidance 

To compare the lower and the upper 95 percent confidence interval of Threat 
Avoidance, the item with the highest perception of workload, with other items, overall 
combat workload item ratings of all pilots were sorted in ascending order of their mean 
values. Tables 4-5a and 4-5b present the sorted data, in which Threat Avoidance has the 
lowest rating value among all combat workload items. The lower mean value of rating 
signified higher level of workload perception. Figure 4-11 illustrates the mean value of 
Threat Avoidance ratings by all pilots, which were significantly lower than mean values 
of all other combat workload ratings. As shown in the figure, the upper and lower 95 
percent intervals of Threat Avoidance and those of In-flight Emergency coincide. In 
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other words, the mean value of Threat Avoidance ratings is significantly lower than mean 
values of all other combat workload ratings, except that of In-flight Emergency ratings. 


Table 4-5a. Lower and Upper 95% Confidence Intervals of Combat Workload Items 


Combat Workload Items 

Lower 95% 

Cl 

Mean 

Upper 95% Significance 
Cl 

1. Threat Avoidance 

2.0900 

2.2146 

2.3392 


2. In-flight Emergency 

2.1814 

2.3059 

2.4304 


3. Crew Incapacitation 

2.3748 

2.6301 

2.8854 

* + ♦ 

4. Fatigue 

2.4816 

2.6301 

2.7786 


5. Adverse Weather 

2.5192 

2.6575 

2.7959 


6. Aircraft Maneuvering 

2.6042 

2.7763 

2.9484 


7. In-flight No-notice 

2.7276 

2.8493 

2.9710 


Mission Changes 

8. Equipment Degradation 

2.8413 

2.9726 

3.1039 


9. Maintaining Situational 

3.1772 

3.3288 

3.4803 


Awareness 

10. Threat Detection 

3.1783 

3.3379 

3.4975 


11. Night Operations 

3.2098 

3.3470 

3.4843 


12. Night Low Level 

3.1352 

3.3562 

3.5772 


Navigation 

13. Terrain Avoidance/ 

3.2830 

3.4658 

3.6485 

♦ + 

Terrain Following 

14. Unfamiliar Terrain 

3.4612 

3.6073 

3.7534 

♦ ♦ + 

15. Command and Control 

3.4558 

3.6101 

3.7644 


16. Low Level Navigation 

3.5681 

3.7397 

3.9114 


17. Managing Radio 

3.6310 

3.7580 

3.8850 

* + * 

Communication 

18. Formation 

3.6556 

3.7854 

3.9151 


Responsibilities 

19. Target Acquisition 

3.6319 

3.8174 

4.0028 


20. Visual Orientation 

3.7357 

3.8940 

4.0524 

+ * 

21. Responding to Ground/ 

3.8793 

4.0047 

4.1301 


Airborne Controller 

Instructions. 
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Table 4-5b. Lower and Upper 95% Confidence Intervals of Combat Workload Items 


Combat Workload Items Lower 95% Cl Mean Upper 95% Cl Significance 


22. 

Mission Planning 

3.9468 

4.1045 

4.2623 


23. 

Management of TOT 

3.9853 

4.1187 

4.2522 


24. 

Shifting Attention to 
Targets of Opportunity 

3.9465 

4.1370 

4.3275 


25. 

Monitoring Flight 
Instruments 

4.1817 

4.3014 

4.4211 


26. 

Crew Coordination 

4.1852 

4.3699 

4.5545 


27. 

Type of Drop 

4.3515 

4.5138 

4.6760 


28. 

Munitions Deployment 

4.3495 

4.5160 

4.6825 


29. 

Refueling Operations 

4.4736 

4.5982 

4.7227 



s 
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Figure 4-11. Range of Lower and Upper 95 % Confidence Interval 
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CWL (Combat Workload’) Data Analysis 


In this section, the overall CWL scores and those of different aircraft types are 
analyzed and the outcomes of the analysis are presented. In Chapter III, CWL was 
described as a variable which measures the overall perception of in-flight workload on a 
combat mission. Overall CWL score for each individual was calculated as a summation 
of their responses to significant combat workload items. CWL score of an individual 
could be as low as zero and as high as 135. A lower value of CWL score means that the 
pilot values his or her in-flight workload on a combat mission relatively higher than 
others. That is, the magnitude of the CWL score of a pilot is inversely related to the 
pilot’s perception of in-flight workload on a combat mission. The frequency distributions 
of CWL are presented in separate diagrams. The results of stepwise regression to predict 
overall CWL score and that of the particular aircraft type are discussed. Finally, CWL 
scores are compared across aircraft types. 

The methods to determine which of the combat workload items make up CWL 
were explained in Chapter III. The results of two methods, the Principal Component and 
Cronbach’s alpha, are presented in Tables D-65a, D-65b and D-66 in Appendix D. Both 
of the SAS statistical software package outputs were reformatted for a better presentation. 
Based on the results of non-rotated factor analysis, the Principal Component, depicted on 
the SAS output in Table D-65, twenty seven out of twenty-nine combat workload items 
were eligible to create the CWL variable. 
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According to results of the Principal Component Analysis, Mission Planing and 
Monitoring Flight Instruments were excluded from the listing of combat workload items 
which make up the overall CWL scores. All combat workload items had a factor loading 
value of 0.3635 or higher. In addition to two combat workload items, the Principal 
Component Analysis pointed out all the remaining items. Questions 38 through 47, as 
survey items which were measuring factors other than that of CWL scores of pilots. 

The results of Cronbach’s alpha correlation analysis are presented in Table 4-6. 
The complete output of the correlation analysis can be found in Appendix D (Table D- 
66). A Cronbach’s coefficient alpha value of 0.92 for both raw and standardized values, 
as indicated in Table 4-6, shows a very high internal consistency of CWL measure (SAS 
Institute, 1990: 214). This value suggests that most of the variance of CWL score is 
error-free, and accounted truly by the CWL score. In Table D-66 “correlation with the 
total values” represent the percentage of correlation of each item with the remaining 
combat workload items that make up the overall CWL score. 

_ Table 4-6. Reliability Test of CWL—Correlation Analysis _ 

Cronbach Coefficient Alpha 
for RAW variables : 0.920502 

for STANDARDIZED variables: 0.922903 


Table 4-7 lists the CWL score statistics by aircraft type, as well as overall CWL 
score statistics. Statistical data were sorted by ascending order of the mean CWL score. 
Based on the average CWL score, it might be concluded that KC-135 pilots’ perception 
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of in-flight workload on a combat mission were the highest of pilots flying other types of 
aircraft. F-16 pilots’ perception of in-flight workload on a combat mission was the 
lowest of all. A casual look at the table also reveals that the mean CWL scores of KC- 
135, C-130, and A/AO-10 aircraft were below the mean of the overall CWL scores, while 
those of B-52, F-15, and F-16 aircraft were higher. Kurtosis and Skewness values show 
that CWL score distributions of all aircraft types are close to normal distribution, as well 
as the CWL score distribution of all pilots. 

Figure 4-12 graphically illustrates the frequency distribution of CWL scores of all 
pilots. Similarly, Figures 4-13 through 4-18 depict the frequency distribution of CWL 
scores of each aircraft group. Normality of distributions in each case can also be seen 
from the graphs of the overall and the particular CWL score distributions. 


Table 4-7. Statistics of Mean CWL Scores By Aircraft 


Aircraft 

Mean 

Std. Dev. 

Minimum 

Median 

Maximum 

Skew 

Kurtosis 

KC-135 

68.359 

13.523 

45 

67.0 

100 

0.2683 

-0.6489 

C-130 

74.719 

15.361 

33 

77.5 

101 

-0.8049 

0.3281 

A/AO-10 

78.588 

12.432 

46 

78.0 

104 

-0.1328 

0.3114 

B-52 

80.385 

12.300 

53 

78.0 

108 

0.0322 

-0.2460 

F-15 

83.773 

17.741 

17 

87.5 

118 

-1.1118 

2.9399 

F-16 

86.161 

15.016 

37 

87.0 

114 

-0.7538 

2.1882 

Overall 

78.635 

15.601 

17 

79.0 

118 

-0.4160 

0.7826 
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Number of Pilots 



Figure 4-12. Distribution of Overall CWL Scores 
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Figure 4-14. Distribution of B-52 Pilots' CWL Scores 



Figure 4-15. Distribution of C-130 Pilots' CWL Scores 











Figure 4-16. Distribution of F-15 Pilots' CWL Scores 



Figxire 4-17. Distribution of F-16 Pilots' CWL Scores 
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Figiire 4-18. Distribution of KC-135 Pilots' CWL Scores 


The stepwise regression routine resulted in the following independent variables 
and the associated coefficient values as shown in Table 4-8. The stepwise regression 
routine used the demographic data mentioned in Chapter III as independent variables to 
predict pilots’ CWL score. 


Table 4-8. Summary of Stepwise Regression Procedure for 
_Dependent Variable -Overall CWL Score 


Variable 

Model R2 Estimate 

F 

Prob.>F 

INTERCEPT 

74.9004 

537.63 

0.0001 

Q4 * 

0.0425 -1.4652 

5.49 

0.0201 

Q8 ** 

0.0682 1.6046 

11.76 

0.0007 

* Q8: What mission did you fly in combat most recently? 

** Q4: What is your current aircraft? 
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The result of the stepwise regression show that the model’s value was as low 
as 0.0682, which means that only 6.82 percent of the variability of pilots’ CWL scores 
can be explained by the two independent variables of the current aircraft flown, and most 
recent combat mission flown. 

Next, a stepwise regression routine on SAS statistical software package was 
performed on the data grouped by aircraft type. Attempts to build a linear model to 
predict the CWL scores of pilots from each aircraft type failed. The summaries of 
stepwise regression routines for aircraft types are presented in Table 4-9. The highest 
of 0.1921 was achieved as result of stepwise regression procedure on C-i30 data, which 
did not constitute a predictive model for CWL scores. Most models for aircraft resulted 
in one independent variable or two as predictors. Therefore, the use of these models, as 
well as the one in Table 4-8, to predict pilots’ perception of in-flight workload on a 
combat mission is not supported. 

Table 4-9. Summary of Stepwise Regression Procedure 

_for Aircraft Types._ 


Aircraft 

Variables 

Model R2 

F 

Prob.>F 

A/AO-10 

Q1,Q8 

0.0840 

2.74 

0.0815 

B-52 

Q1,Q6 

0.1777 

3.67 

0.0360 

C-130 

Q6 

0.1921 

6.66 

0.0154 

F-15 

Q5 

0.1284 

4.72 

0.0374 

F-16 

Q6 

0.1284 

4.72 

0.0374 


KC-135 No Variable met the significance level of 0.015. 

Q1: What is your current rank? 

Q5: Were you qualified in another aircraft previously? 

Q6: How much combat time did you accumulate? 

Q8: What mission did you fly in combat most recently? 


4-49 





One-way Analysis of Variance (ANOVA) procedures were performed on the 
independent variables which were included in stepwise regression. In analysis of 
variance, one is interested in testing whether a group of two or more means are equal 
(Elliot, 1995: 87). ANOVA procedures are conducted before Bonferroni procedures are 
performed. The one-way ANOVA procedure assures that a factor in a linear model is 
significant, that is, the means for the levels of that factor are equal (Elliot, 1995: 87). The 
results of the ANOVA procedures are reported in Table 4-10. All factors, independent 
variables, were found significant at the 95 percent significance level. 


Table 4-10. Summary of One-Way ANOVA Procedure 

forCWL 


Variable df F Prob.>F 


Q4 

5 

7.17 

0.0010 

Q6 

6 

2.27 

0.0384 

Q8 

7 

5.49 

0.0010 


Q4: What is your current aircraft? 

Q6: How much combat time did you accumulate? 

Q8: What mission did you fly in combat most recently? 


Finally, the CWL scores were examined in terms of differences in mean CWL 
scores across the aircraft types. In Table 4-7 CWL score statistics were given including 
the mean CWL scores. The data were basically ordered in ascending order of the mean 
CWL score of each aircraft. The Bonferroni procedure provides multiple comparisons of 
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mean CWL scores. The Bonferroni procedure was performed on SAS statistical analysis 
software package at the 95 percent confidence level. Table D-67 in Appendix D presents 
the results of Bonferroni procedure. Table D-67 shows significant differences in mean 
comparison of the six aircraft types surveyed, and pairs of aircraft with a significant 
difference between them are indicated by triple-asterisks. The degree of pilots’ 
perception of in-flight workload on a combat mission is showed in Table 4-11. 

The results of the Bonferroni procedure reveal that there was no aircraft type in 
which pilots’ perception of in-flight workload on a combat mission is significantly 
different than those of the remaining aircraft types. Mean CWL scores of pilots from 
each aircraft type were very close to each other. This can easily be noticed when 
Table 4-11 is examined closely. For instance, mean CWL scores of A/AO-10 pilots 
indicated that the in-flight workload perception of pilots varies between Level I, the 
lowest level of perception, and Level 4, the second higher level. Despite the significant 
indifference in means, KC-135 and C-130 pilots’ perception of in-flight workload was 
rated at the highest level. In fact, C-130 pilots also appeared in lower three levels. 
Degrees II, III and IV. However, the table shows that KC-135 pilots’ perceptions of in¬ 
flight workload were significantly different from those of pilots from four aircraft, F-16, 
F-15, B-52 and AO/A-10. This research examined only the mean scores of pilots from 
six aircraft t3^s. A study including more aircraft types might reveal different ratings of 
pilots’ perceptions regarding in-flight workload in combat. 
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Table 4-11. Summary of CWL Means by Aircraft— Pilots' 
Perception of In-flight Workload on a Combat Mission 


Degree of In-flight 
Workload 

Aircraft Types 

I (Lowest) 

F-15/F-16/B-52/A-10 

II 

F-16/B-52/A-10/C-I30 

III 

B-52/A-10/C-130 

IV 

A-lO/C-130 

V (Highest) 

C-I30/KC-135 


Chapter Summary 

Examination of the personal characteristics of pilots who participated in this 
research revealed that the sample group consisted of highly experienced pilots. Based on 
the criteria explained in Chapter III, one out of seven pilots had accumulated considerable 
amoimt of total USAF and combat flying time to be considered an “experienced” pilot. 
Most of the pilots had accumulated more than 1,000 hours of flying time, and at least 
some combat time. Almost all of the pilots had flown in combat. Nearly all of them had 
flown their current aircraft in combat. 

The pilots’ responses regarding combat workload items brought valuable insights 
to in-flight workload in combat and new facts regarding workload assessment. Pilots’ 
perceptions of combat workload items revealed that no combat workload item was 
believed to increase pilot workload “dangerously” on a combat mission. However, they 
agreed that some combat workload items were likely to increase in-flight workload in a 
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distracting manner. Regardless of aircraft or missions flown. Threat Avoidance and In¬ 
flight Emergency were rated consistently higher among the combat workload items. 

Crew Incapacitation, Adverse Weather, Fatigue and Aircraft Maneuvering were other 
highly-perceived items. A worst case scenario for increased workload in a combat 
mission could be predicted as an in-flight emergency (or multiple emergency) occurring 
while avoiding a known enemy threat. The scenario could become more difficult if one 
or more of the other items mentioned above were added in. Nevertheless, differences in 
combat workload appeared in lower levels of importance when they were examined 
across different aircraft and mission types. 

The analysis of CWL scores of pilots revealed that the mean CWL scores were 
distributed normally, both in general, and across different aircraft types. KC-135 pilots 
perceived in-flight workload on a combat mission significantly higher than pilots of other 
aircraft. However, the differences between mean CWL scores of pilots from six aircraft 
types were not significant; that is, mean CWL scores were very close to each other. 
Attempts to predict CWL scores based on the selected personal characteristics data via 
stepwise regression routine on SAS statistical analysis software package failed to produce 
significant findings. 

In the next chapter, the findings of written material regarding in-flight workload 
assessment siunmarized in Chapter II, and the results of analysis of the data on pilots’ 
perceptions of in-flight workload are blended in an effort to answer the questions 
introduced in Chapter I. 
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V. Conclusions and Recommendations 


Chapter Overview and Introduction 

In this chapter, two bodies of information, the review of the relevant literature in 
Chapter II, and the results of analysis of the survey data presented in Chapter IV, are 
integrated. By doing so, the main research question and the investigative questions, with 
their respective hypotheses, are answered. 

Answers to each investigative questions are discussed in separate sections in this 
chapter. Depending on the nature of the questions, the discussion is detailed by aircraft 
or mission t 3 q)e flown. Answers to each question are derived from the results of data 
analysis, and they are supported by the findings of literature review and the related 
appendices when appropriate. Building on the answers to investigative questions, 
response to the main research question is provided. 

Also in this chapter, recommendations for further researches are provided. These 
recommendations should provide insights to those individuals who want to take this 
research effort forward. 

One appendix of this research project is devoted solely to the comments of pilots 
who provided their concerns and opinions. Appendix B, Survey Comments, contains 
representative opinions of pilots on in-flight workload assessment. Those pilots took 
extra time to include their comments after completing a prolonged survey. 
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Investigative Question!; What pilot workload measurement systems have been 


developed that might pertain to a high-threat combat environment? 

An extensive review of literature found that there were four major group of 
workload assessment techniques: (1) Primary Task Performance Techniques, (2) 
Secondary Task Performance Techniques, (3) Physiological Measurement Techniques, 
and (4) Subjective Techniques. However, reviewing these techniques revealed that there 
is no suitable method of measuring workload in an aircraft while performing high threat 
combat missions. As Neville et al stated, unfortunately, no suitable methodology for 
measuring pilot performance in a wartime setting could be found (Neville et al, 1994: 
346). The implementation requirements of most in-flight workload measures made it 
difficult to use them in the real world environment. Specifically, the use of these 
techniques in a combat environment with an existing enemy threat becomes almost 
impossible due to intrusiveness of the techniques and their impact on mission 
effectiveness. Most of the pilots who selected to write comments in response to Question 
46a, which asked pilots to provide reasoning for not preferring to volunteer workload 
assessment experiments in combat, stated that combat environment is not the place for 
experimenting, or collecting data for empirical studies. A Major with 2501-3000 total 
flying hours and less than combat flying hours stated that “A combat sortie is 
inappropriate for testing. Where would the emphasis be—data collection or bombs on 
target? Would aircrews be required to perform nonstandard maneuvers, or fly in test 
modified aircraft? If the data collection was completely transparent, it might be 
acceptable.” His comments point out that workload studies might be acceptable, but the 
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associated high risk, and cost of the study in terms of mission effectiveness should be 
considered. 

Use of advanced simulators in assessing the workload in an high-threat 
environment could be considered; however, the effects of real threats are difficult to 
create realistically in a simulation world. The answers to a survey question regarding the 
effectiveness of simulators in creating realistic combat workload scenarios revealed some 
interesting information. Almost 2/3 of pilots (61.9 percent) disagreed that simulator 
missions flown in peace time adequately simulate the amount of workload that a pilot 
would experience in combat. Nearly 1/3 of pilots (28.6 percent) did not want to 
participate in experimental studies which measure in-flight workload in a simulated 
combat missions. Most pilots wanted to be excused from such experimental studies, 
because of lack of time, while others commented that simulator missions carmot 
realistically duplicate the combat environment. 

In light of information gathered from the survey, and relevant literature, no pilot 
workload measurement system has been found as a proper means for assessing the in¬ 
flight workload in a high-threat combat environment. 
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Investigative Question 2: Is it possible to determine, from experienced pilots, what 
items would be most important in measuring workload in a high-threat 
environment? 

To answer this question, 29 combat workload items were listed in the survey 
questionnaire, and pilots were asked to evaluate each item in terms of the degree to which 
they believed likely to increase the pilot workload in a combat environment. 

Based on the average values of pilots’ ratings, none of the combat workload items 
was foimd in itself to be “dangerously” important in regard to increasing pilot workload 
in a combat environment. From the list of 29 combat workload items, two combat 
workload items were valued as “distractingly” important. Threat Avoidance and In-flight 
Emergency (Table IV-2) were two combat workload items which pilots believed were 
likely to increase pilot workload “distractingly.” Eleven combat workload items were 
rated as “moderately” important. They were, in descending order of importance Fatigue, 
Crew Incapacitation, Adverse Weather, Aircraft Maneuvering, In-flight No-notice 
Mission Changes, Equipment Degradation, Maintaining Situation Awareness, Threat 
Detection, Night Operations, Night Low Level Navigation, and Terrain Avoidance or 
Terrain Following (Table IV-3). The remaining sixteen combat workload were valued as 
“somewhat” important in terms of increasing pilot workload in combat. 

By surveying a sample of 219 pilots, in which 98.2 percent had combat 
experience, it was possible to determine what items would be most important in 
measuring workload in a high-threat environment. Future researchers of the issue should 
concentrate on the worst scenario of combat workload, which appears to be the case of 
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having an emergency on the aircraft while avoiding an enemy threat. A more challenging 
scenario could occxir when additional combat workload items are introduced into the 
picture. The designers of future aircraft should also keep this finding in mind, and they 
should concentrate their efforts on developing systems to improve pilot workload in such 
scenarios. 


Investigative Question 3; Can anv kind of consensus be reached to determine the 
relative priorities of workload items? 

In an attempt to answer this particular question, the issue was further divided into 
four sub-investigative questions and respective hypothesis. First, answers to these four 
sub-questions were obtained by examining each hypothesis. Then, a final conclusion was 
reached on the answer of the third investigative question. 


Investigative Question 3a: What do pilots of different aircraft type believe are the 
critical items that will increase their workload in a high threat combat 
environment? 

It was hypothesized that pilots’ perceptions of each aircraft type pertaining to the 
combat workload items would differ due to different characteristics of aircraft types. 
From a list of twenty-nine combat workload items, pilots from six aircraft types rated 
each workload item on a five-point scale ranging from “dangerously” to “a little” or “not” 
important. To eliminate the workload items which were not applicable to mission type or 
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aircraft flown, a “not applicable” selection was added to the list of choices. Surprisingly, 
as it was in overall ratings of pilots, no single combat workload item was consistently 
rated as “dangerously” important based on data segregated by aircraft type. In other 
words, pilots from six aircraft type think that no combat workload item “dangerously” 
increases in-flight pilot workload on a combat mission. 

AO/A-10 . Based on the average ratings of 34 AO/A-10 pilots surveyed, five 
combat workload items were found to be “distractingly” important (Table IV-2). They 
were Threat Avoidance, In-flight Emergency, Adverse Weather, Aircraft Maneuvering, 
and In-flight No-notice Mission Changes; among these five. Threat Avoidance was rated 
as the most important combat workload item. AO/A-10 pilots rated 14 combat workload 
items as “moderately” important items in workload assessment (Table D-5). Nine 
workload items were found to be “somewhat” important combat workload items (Table 
D-6). On the average, AO/A-10 pilots rated Crew coordination as of “little” or “no” 
importance, due to fact that they fly mostly single-seat models (Table-7). No combat 
workload item was evaluated as “not applicable” on the average. Night Operations and 
Night Low Level Navigation, as the top two “moderately” important items, might imply 
that AO/A-10 are not well-equipped for night flying. Furthermore, in valuing five items 
as “distractingly” important, which is almost twice as many as other two fighter type 
aircraft, pilots could be pointing out the technological differences in design of AO/A- 
10s, and F-15s and F-16s. 

B-52 . Six combat workload items found to be “distractingly” important by 40 B- 
52 pilots (Table IV-2). These combat workload items were Crew Incapacitation, Threat 
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Avoidance, Fatigue, Night Low Level Navigation, In-flight Emergency, and Aircraft 
Maneuvering. Among all items. Crew Incapacitation was rated as the most important 
combat workload item. B-52 pilots’ average ratings identified 14 combat workload items 
as “moderately” important items in combat workload assessment (Table D-9). According 
to the average rating values of B-52 pilots, the remaining nine combat workload items 
were determined as “somewhat” important where Mission Planning was specified as the 
least important combat workload item (Table D-10). No combat workload item was 
found to be of “little” or “no” importance, an there was no “not applicable” item. The 
design characteristics of B-52 play an important role in pilots determining the six 
“distractingly” important combat workload items. Since crew cooperation is vitally 
important. Crew Incapacitation was rated as the most important item in terms of 
increasing pilot workload. Fatigue is important, because most B-52 aircrew fly many 
hours on a sortie. B-52 pilots are also involved in many missions requiring them to fly 
Night Low Level Navigation, which is also present among six “distractingly” important 
items. In general, the selection of all 29 combat workload items in the survey was proper 
for this particular aircraft, because no workload item was valued as “not applicable” on 
the average. B-52 pilots selected the greatest ntimber of combat workload items, as 
“distractingly” important among all aircraft, that is six items. This might imply that, in a 
high-threat environment, the mission requirement of the B-52 aircraft could create 
circumstances where more factors could cause task saturation. 

C-130 . Beised on the evaluation of 32 C-130 pilots, five combat workload items 
were identified as “distractingly” important (Table IV-2). Among these five combat 
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workload items—which included Threat Avoidance, In-flight Emergency, Adverse 
Weather, Crew Incapacitation, and Fatigue—Threat Avoidance was evaluated as the most 
important combat workload item. Fifteen combat workload items evaluated as 
“moderately” important items in workload assessment according to C-130 pilots’ average 
ratings (Table D-12). Pilots of this type specified six combat workload items as 
“somewhat” important (Table D-13). Shifting the Attention to Targets of Opportunity 
was valued as of “little” or “no” importance (Table D-14). Munitions Deployment and 
Refueling Operations were found to be “not applicable” to C-130 combat operations 
(Table D-13). Fatigue and Crew incapacitation were present in the top listing of combat 
workload items for C-130 aircraft, as they were for B-52 aircraft. As in AO/A-lOs, 
Adverse Weather was rated in the top three important item. This might imply that both 
aircraft are not equipped with devices to fly in all-weather conditions. The survey was 
successful in detecting the two items. Munitions Deployment and Refueling Operations, 
as “not applicable” items, because most C-130 aircraft are not furnished with these 
capabilities. 

F-15 . Three workload items. In-flight Emergency, Crew Incapacitation, and 
Threat Avoidance, were measured as “distractingly” important (Table IV-2). Thirty F-15 
pilots’ average ratings determined nine combat workload items as “moderately” 
important items in combat workload assessment (Table D-17). According to the average 
rating values, the remaining 15 combat items were found to be “somewhat” important 
when assessing workload (Table D-18). F-15 pilots’ choice of the least important 
combat workload item was Air Refueling Operations. No combat workload item was 
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found to be of “little” or “no” importance. Although almost two thirds of the combat 
workload items were rated as “not applicable” item by at least one pilot, none of them 
was valued as “not applicable” on the average. Crew Incapacitation was present among 
three “distractingly” important items. This could be due to the fact that sample of this 
surveys contained several pilots flying F-15E models which are two-seat versions of F-15 
aircraft. Crew cooperation is important in mission success in F-15E aircraft. 

F-16 . Only two combat workload items, In-flight Emergency and Threat 
Avoidance, were valued as “distractingly” important by 44 F-16 pilots (Table IV-2). 
Twelve combat workload items were found as “moderately” important items in combat 
workload assessment (Table D-20). Based on the average rating values, thirteen F-16 
“somewhat” important combat workload items were determined. Air Refueling was 
valued as of “little” or “no” importance. Crew Coordination was rated as “not 
applicable” to F-16 aircraft due to fact that all F-16s, except training models, are single- 
seated fighter aircraft. On the other hand. Crew Incapacitation was rated as the last item 
on “the moderately” important item listing. Pilot might infer the case as themselves were 
incapacitated. F-16 pilots only two items as “distractingly” important; this might imply 
that the F-16 aircraft is equipped with advanced technologies which improves in-flight 
workload. 

KC-135 . Three combat workload items, which are foxmd to be “distractingly” 
important by 39 KC-135 pilots, were Crew Incapacitation, In-flight Emergency, and 
Threat Avoidance (Table IV-2). KC-135 pilots valued six combat workload items as 
“moderately” important items in combat workload assessment (Table D-25). According 
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to the average rating values, twelve “somewhat” important combat workload items for 
KC-135S were determined (Table D-26). On the average, four combat workload items 
were valued as of “little” or “no” importance items. Target Acquisition, Type of Drop, 
and Munitions Deployment were found “not applicable” to KC-135 aircraft. As in all 
multi-seat aircraft. Crew Incapacitation was present among the top listings in KC-135 
aircraft. 

When survey data were analyzed, in general, it was found that the hypothesis was 
proved to be correct. When combat workload items of each aircraft in lower levels of 
importance were reviewed the hypothesis was most strongly supported. However, at the 
highest level of importance, when “distractingly” important combat workload items’ 
listing was reviewed, the hypothesis appeared to fail. In the listings of combat workload 
items for each aircraft type. Threat Avoidance and In-flight Emergency were present, and 
they were rated as likely to increase pilot workload “distractingly” in combat. Threat 
Avoidance was present in all aircraft types within top three listings, and it was rated as 
the most important item in two aircraft types. In-flight Emergency was present in all 
aircraft listings within the top five combat workload items, and it was rated as the most in 
two aircraft types. Other common combat workload items found “distractingly” 
important across aircraft types were Crew Incapacitation, Fatigue, Adverse weather, and 
Aircraft Maneuvering. Among these combat workload items. Crew Incapacitation was 
rated as the most important item in two aircraft type. Therefore it is unfair to state that 
the hypothesis hold truth in the category of “distractingly” important combat workload 
items. 


5-10 



Despite the lack of commonality in lower levels of importance, it might be 
concluded that, regardless of aircraft type, the worst scenario of pilot workload in combat 
exists when an in-flight emergency occurs while avoiding a known enemy threat. It can 
be inferred that the selection of workload items was proper, because it represented 
common items of a combat mission for six aircraft types surveyed. AO/A-10, B-52 and 
C-130 pilots rated more items as “distractingly” important than pilots of F-15, F-16, and 
KC-135 aircraft. Technologies in the latter group of aircraft might be a factor in lower 
number of items in this category. 

Finally, the answer to Investigative Question 3a could be concluded as the level of 
workload perceived by pilots changes depending the type of the aircraft flown; however, 
for those combat workload items with “distractingly” important rating, the level of 
workload perceived by pilots remains almost same by the type of the aircraft flown. 


Investigative Question 3b: Does the level of workload perceived bv pilots change 
depending on the type of the mission flown? 

To investigate the answer the question above it was hypothesized that the ratings 
of combat workload items of pilots would change depending on the type of the mission 
flown. The entire data on twenty-nine combat workload items were segregated by 
mission type flown most recently. Strategic Deterrence was flown by only two of the 
pilots. Therefore, along with the data pertaining to the “other” category of mission flown, 
the Strategic Deterrence mission data were exempt fi-om the analysis. Surprisingly, as 
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was trae in overall ratings of pilots, and in ratings by aircraft type, no combat workload 
item was rated as “dangerously” important based on data segregated by mission type 
flown. The results of the analysis of combat workload items for each mission type are 
summarized below. Emphasis is given those workload items which were likely to 
increase pilot workload “distractingly.” 

Conventional Heavy Bombing Missions . Thirty-six of pilots surveyed flew 
Conventional Heavy Bombing Missions in combat most recently. All pilots of this group 
flew the specified mission in B-52 aircraft. Based on the mean values of pilots’ ratings, 
six combat workload items were identified as “distractingly” important in combat (Table 
IV-3a). They were Threat avoidance. Crew incapacitation. Fatigue, Night Low Level 
Navigation, and In-flight Emergency. Among these six combat workload items. Threat 
Avoidance was valued as the most important item. Fifteen combat workload items were 
evaluated as “moderately” important (Table D-30). According to the average rating 
values of pilots of this mission type, nine combat workload items were identified as 
“somewhat” important (Table D-31). On the average, pilots of this mission type chose 
Mission Planning as the least important combat workload item. No combat workload 
item was found to be of “little” or “no” importance, and there were no “not applicable” 
items. 

Theater Airlift Missions . Thirteen pilots most recently flew Theater Airlift 
Missions in combat. All pilots of this group flew the particular mission in C-130 aircraft. 
Based on the mean values, four combat workload items were identified as “distractingly” 
important in combat (Table IV-3a). These combat workload items were Threat 
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Avoidance, In-flight Emergency, Crew Incapacitation, and Adverse Weather. Fourteen 
combat workload items that were evaluated as “moderately” important (Table D-33). 
Pilots flying Theater Airlift missions in combat specified nine “somewhat” important 
combat workload items (Table D-34). No combat workload item was found to be of 
“little” or “no” importance. Munitions Deployment and Air-to-Air Refueling Operations 
were identified as “not applicable” items (Table D-30). 

Tactical Airlift Missions . Eighteen of the pilots surveyed flew Tactical Airlift 
Missions in combat. Like pilots flew Theater Airlift missions, all pilots in this group also 
flew the particular mission in C-130 aircraft. Pilots’ ratings of this mission type valued 
three combat workload items as “distractingly” important (Table IV-3a). Threat 
Avoidance, Fatigue and In-flight Emergency were the three combat workload items with 
“distractingly” important rating. Eighteen combat workload items that were evaluated as 
“moderately” important (Table D-38). Like the pilots flying Theater Airlift mission, the 
pilots of this mission type indicated Shifting Attention to Targets of Opportunity was 
specified as of “little” or “no” importance (Table D-40). Munitions Deployment and 
Air-to-Air Refueling Operations were identified as “not applicable” items (Table D-41). 

Air-to-Air Refueling Missions . Thirty-seven pilots who most recently flew Air- 
to-Air Refueling missions were KC-135 pilots. Pilots of this mission type rated three 
combat workload items as “distractingly” important. They were Crew Incapacitation, In¬ 
flight Emergency, and Threat Avoidance. Six combat workload items that were valued as 
“moderately” important are listed in Table D-43. Pilots flying this mission type 
identified fourteen items as “somewhat” important combat workload items (Table D-44). 
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Five combat workload items were considered as of “little” or “no” importance workload 
items (Table D-45). Target Acquisition, Type of Drop and Munitions Deployment were 
found to be “not applicable” to the particular mission type (Table D-46). 

Air Supe riority Missions . Six F-15 and two F-16 pilots flew Air Superiority 
missions in combat. Pilots of this mission type rated four combat workload items as 
“distractingly” important. They were Fatigue, Threat Avoidance, In-flight Emergency, 
and Adverse Weather. Seven combat workload items were identified as “moderately” 
important (Table D-48 ). Thirteen of the remaining combat workload items were 
evaluated as “somewhat” important (Table D-49). Management of TOT, Responding to 
Ground or Airborne Controller Instructions, Refueling Operations, and Type of Drop 
were four combat workload items considered as of “little” or “no” importance (Table D- 
50). Crew Coordination was found to be “not applicable” to the particular mission type 
(Table D-51). 

Close Air Support . Twenty-five of 26 pilots who most recently flew Close Air 
Support missions in combat were AO/A-10 pilots, along with one F-16 pilot. Pilots of 
this mission type rated four combat workload items presented as “distractingly” 
important. These combat workload items were Threat Avoidance, In-flight Emergency, 
Adverse Weather, and Aircraft Maneuvering. Thirteen combat workload items were 
identified as “moderately” important (Table D-53). Ten combat workload items were 
found to be “somewhat” important (Table D-54). One remaining combat workload item. 
Crew Coordination was valued as of “little” or “no” importance (Table D-55). No 
combat workload item was found to be “not applicable” to this mission type. 
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Air Interdiction . Among the 54 pilots who most recently flew Air Interdiction 
missions in combat were six AO/A-10, two B-52, twenty-four F-15, and twenty-two F-16 
pilots. Pilots’ ratings of this mission type identified two combat workload items as 
“distractingly” important, In-flight Emergency and Threat Avoidance (Table IV-3b). 
Twelve combat workload items were identified as “moderately” important (Table D-57), 
There were fifteen combat workload items with “somewhat” important workload ratings 
(Table D-58). Crew Coordination was found to be of “little” or “no” importance (Table 
D-59). No combat workload item was valued as “not applicable” to this mission type. 

Combat Air Patrol . Ten pilots most recently flew Combat Air Patrol missions in 
combat. Among those were one AO/A-10, one F-15, and eight F-16 pilots. Pilots flying 
this mission type rated two combat workload items as “distractingly” important, In-flight 
Emergency and Threat Avoidance (Table IV-3b). Nine combat workload items were 
identified as “moderately” important (Table D-61). Pilots of this mission type identified 
fifteen combat workload items as “somewhat” important (Table D-62). Management of 
TOT and Monitoring Flight Instruments were rated as of “little” or “no” importance 
(Table D-63). Crew Coordination was evaluated as “not applicable” to this mission type 
(Table D-64). 

It was found that the hypothesis holds true, especially when combat workload 
items of each aircraft are reviewed in lower levels of importance. However, when 
“distractingly” important combat workload items are investigated, the hypothesis is 
tentative. Threat Avoidance and In-flight Emergency were rated “distractingly” 
important within each mission group. Threat Avoidance was rated highest in four of eight 
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mission groups evaluated, and it was present within the top three listings of all mission 
types. In-flight Emergency was present within the top five listings of all mission groups, 
where it was rated as the most important item in two mission groups. Adverse Weather, 
Aircraft Maneuvering, Crew Incapacitation, and Fatigue were the other common 
“distractingly” important combat workload items. The main reason for such 
commonality stems from the fact that certain types of missions are flown by certain 
aircraft. For instance, KC-135 aircraft combat workload item categories are those of Air- 
to-Air Refueling missions; and, B-52 aircraft combat workload item categories and those 
of Conventional Heavy Bombing missions are the same. However, the combat workload 
items’ listings of missions which are flown by several different aircraft types differ from 
those of individual aircraft type. For instance, although Fatigue is not present in the 
listing of “distractingly” important combat workload items of F-15 or F-16 aircraft, it is 
the top item in the listing of Air Superiority missions’ “distractingly” important combat 
workload items. Although Tactical Airlift and Theater Airlift are flown on the same type 
of aircraft, the C-130 Hercules, the listings of combat workload items varies between 
these two mission types. On both missions the survey successfully distinguishes the 
same “not applicable” items. These facts show that the survey is able to distinguish 
among the workload items for different types of missions when the same missions were 
flown on different aircraft, or two missions are flown on the same type of aircraft. The 
combat workload item categories of other mission types are slightly different than those 
of aircraft in which particular missions were flown in combat. 
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Consequently, the answer to Investigative Question 3b could be concluded as the 
level of workload perceived by pilots changes depending the type of the mission flown; 
however, for those combat workload items with “distractingly” important rating, the level 
of workload perceived by pilots does not change by the type of the mission flown. 


Investigative Question 3c: Does the level of workload perceived bv pilots of each 
type vary bv the experience level of the pilots? 

It was hypothesized that pilots with more flight experience would be likely to 
specify higher ratings (lower perception of workload) for the combat workload items. To 
compare the average ratings of “experienced” and “inexperienced” pilots, average combat 
workload items’ ratings were evaluated separately based on the criteria mentioned in 
Chapters III and IV. After segregation of the data, it was found that data set contained 30 
experienced and 189 inexperienced pilots. To test the hypothesis, two-tailed t-tests were 
performed on each pair of combat workload items listed in Tables IV-4a and IV-4b. 
However, at the 95 percent confidence level, only Managing Radio Commvmication was 
significantly rated lower among “experienced” pilots in comparison to the 
“inexperienced” ones. Definitely, the hypothesis associating the lower rate of combat 
workload perception with increased level of experience is not supported. Thus, the 
answer to Investigative Question 3c is that the level of workload perceived by pilots of 
each type does not vary by the experience level of the pilots. 
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Investigative Question 3d: Is there a particular combat workload item that 
significantly possesses higher perception bv all pilots? 

It was hypothesized that the workload of “Threat avoidance” would be perceived 
as of greater significance than the other combat workload items by all pilots. Higher 
perception of workload was indicated by a lower value of mean rating according to Likert 
scale ratings used in the questionnaire (Appendix A). As stated in the previous chapter, 
Threat Avoidance was rated significantly lower than all other combat workload items 
except for In-flight Emergency at the 95 percent confidence level. The listing of 29 
combat workload items in Tables IV-5a and IV-5b contains the lower and the upper 
values of mean scores at 95 percent significance level. Triple-asterisks indicate items 
whose mean values are significantly different than that of Threat Avoidance. Mean 
values of all items are significantly different than that of Threat Avoidance, except In¬ 
flight Emergency. As also shown graphically in Figure IV-11, lower and upper 95 
percent interval ranges of Threat Avoidance intersect with those of In-flight Emergency. 
Therefore, this hypothesis is not supported although pilots’ perception of workload of 
Threat Avoidance was highest of all combat workload items. Thus, the answer for 
Investigative Question 3d is that there is no particular workload item that posses 
significantly higher rating. 

By reviewing the answers to all four sub-investigative questions, in general, it 
might be concluded that no consensus could be reached as to the relative priorities of 
workload items. However, when combat workload items are examined within the 
“distractingly” important category. Threat Avoidance and In-flight Emergency stand out 
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as the two of greatest importance in pilot workload assessment. The worst workload 
matrix would consist of Threat Avoidance and In-flight Emergency scenarios existing 
concurrently. 


Investigative Question 4: Could it be possible to enumerate combat workload 
perceptions of pilots overall, or those of pilots from different aircraft depending 
upon their characteristic data? 

To answer this investigative question it was hypothesized that a model could be 
developed to predict combat workload perceptions of pilots in general, or those of pilots 
from particular aircraft by using pilot demographic data as independent variables. 

The distribution diagrams describing Combat Workload (CWL) scores of the 
entire data set and those of different aircraft types were examined (Figure IV-12 through 
IV-18). CWL scores of individual aircraft were normally distributed, as well as that of 
the entire data Attempts to develop a regression model that would predict CWL scores 
of pilots failed. In other words, the pilots’ perception of in-flight workload on a combat 
mission could not be predicted by use of a regression model. Attempts to build similar 
models for each aircraft also failed. 

However, the Bonferroni procedure was performed among the CWL scores of six 
aircraft types. The results showed that differences in means of the CWL scores for six 
aircraft were not significant; instead, they were very close in values. Despite the lack of 
significant difference among the means, KC-135 pilots’ perception of in-flight workload 
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on a combat mission was significantly the higher than pilots from four other aircraft 
types, except C-130 pilots (Table IV-11). The similarities of mean values of CWL scores 
among pilots from six aircraft prevent us making broad statements about the conclusion 
of the Bonferroni procedure. It was surprising to find that a higher degree of perception 
of in-flight workload on combat missions was indicated by KC-135 pilots, because the 
number of KC-135 “distractingly” combat workload items was only three, fewer than that 
of four other aircraft types. Furthermore, KC-135 aircraft seldom fly missions in a high 
threat environment. They usually perform refueling missions over friendly territory. 
Because they are expensive and strategically important weapon systems, these aircraft are 
defended and escorted by advanced fighter aircraft resulting in lower risk of an enemy 
attack. Also, they usiially refuel fighter packages at higher altitudes where the risk of 
enemy groimd fire is nearly zero. Another surprising finding of the Bonferroni procedure 
was that F-16 pilots were in the lower segment of the workload perception scale. Perhaps 
this finding results from the fact that advanced technologies improve in-flight workload, 
although advanced technologies integrated into the F-16 cockpit demand higher 
performance from pilots who must accomplish their tasks by themselves. Like F-16 
pilots, F-15 pilots appeared in the lower portions of the scale. 

The research data overall do not support the hypothesis that pilots’ perception of 
in-flight workload could be predicted by use of regression model. 

In light of the answers to investigative questions, the answer to the main research 
question, “Based on the pilot experience, is it possible to assess pilot workload in a high 
threat combat environment?” could be concluded as follows: Although the findings of the 
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research data have brought many insights to workload assessment in combat or high- 
threat environment, it is highly optimistic to believe that in-flight workload in such an 
environment can assessed by the help exclusively of perceptions of experienced pilots. 


Further Research 

This research effort suggests future research in the following areas: 

1. A follow-up of this study could be conducted among pilots of new weapon 
systems including B-1, C-17, C-141, F-111, and F-117 aircraft. A revised version of this 
survey could be administered to those pilots mentioned above to determine the impact of 
developed aircraft technologies on the pilots’ perception of in-flight workload. The result 
could increase the validity of this research effort. 

2. A similar study could investigate the degrees of in-flight workload perceived 
by pilots on different segments of combat missions. Combat workload items could be 
grouped imder different mission segments such as take off and landing, en-route to the 
target area, within the target area, egress, and en-route home. Such a study might bring 
new insights to in-flight workload assessment in high-threat environment. 

3. A follow-up study could be conducted among pilots who participate in 
training exercises such as FLAG and JRTC exercises. These exercises simulate the 
combat environment to a great extent, bringing on increased level of aircrew fatigue and 
in-flight workload. 
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Summary 


The overall goal of this chapter was to answer the main research question 
introduced in Chapter I. The research question was: Based on the pilot experience, is it 
possible to assess pilot workload in a high threat combat environment? To determine the 
answer to main research question, four investigative questions were posed. First, answers 
to these investigative questions were found by use of respective hypotheses. Findings of 
Chapter IV and information gathered from previous chapters and related appendices were 
used to answer these investigative questions. In light of the answers to investigative 
questions, it was concluded that it would be highly optimistic to state that it was possible 
to assess in-flight workload in a combat or high-threat environment based on perceptions 
of experienced pilots. 
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Appendix A: The Cover Letter and The Survey Questionnaire 


MEMORANDUM FOR. 

ATTENTION: 


FROM: AFIT/LAC 

2950 P Street 

Wright-Patterson AFB, OH 45433-7765 

SUBJECT: Survey Package of Pilots’ Perception of Workload Assessment in Combat or 

High-Threat Environment-ACTION MEMORANDUM 

1. Please take the time to complete the attached questioimaire and return it in the 
enclosed envelope by 15 September 1997. 

2. The survey measures your perceptions of workload assessment in a high threat 
combat environment. The data we gather will become part of an AFIT research 
project and may influence future aircraft design and workload assessment techniques. 
Your individual responses will stay anonymous; and, combined with others, they will 
not be attributed to you personally. 

3. This survey is unclassified, and all responses to the survey must be imclassified, so 
that the results can be releasable to the public domain. 

4. Your participation is completely volvmtary, but we would certainly appreciate your 
help. For further information, contact me at DSN 785-1213. 


Dr. David K. Vaughan 
Associate Dean, AFIT/LAC 
Air Force Institute of Technology 
Graduate School Acquisition and 
Logistics Management 
Email: dvaughan@afit.af.mil 
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INSTRUCTIONS 


This survey is designed for U.S. Air Force officers with the aeronautical rating of 
PILOT, SENIOR PILOT, or COMMAND PILOT who have flown a single or multi- 
engine aircraft in combat or in a “combat environment” as described in the survey 
questionnaire. The survey should not be filled by any personnel other than those 
specified above. 

This questionnaire consists of 53 items. All items except items 4,5,7,37a, 44a, 
45a, 46a, 47a and 48 are to be answered on the enclosed answer sheet, AFIT Form 11C. 
The remaining six questions are to be answered on the questionnaire booklet. If you 
select the “other” choice as an answer for applicable questions, please specify your 

answer in the space provided (x.(specify)) in the questionnaire booklet 

after you have marked the “other” choice on the answer sheet. 

The answer sheet and the questionnaire booklet should be mailed in the pre¬ 
addressed envelope provided NLT 15 September 1997 . 

Please use a No. 2 lead pencil, and remember the following points: 

1. Make heavy black marks 

2. Erase neatly when needed. 

3. Make no stray markings. 

4. Do not staple, fold or tear the response sheet. 

We remind you NOT to fill in your name on any of the papers so that your answer 
Avill remain anonymous. 


USAF Survey Control Number (SCN): 97-52 
Expiration Date: 31 October 1997 





Survey Questionnaire 

USAF PILOT SURVEY OF FLIGHT WORKLOAD IN A COMBAT 
ENVIRONMENT 

For the purpose of the present study the “combat environment” is 
defined as any flight environment where the USAF aircraft are likely to be 
exposed to any type of non-friendly grovmd or enemy defense systems capable 
of firing. In parallel to the definition above, “combat flights” are defined as 
those flights conducted in any “combat environment.” Peacetime training 
flights which are conducted to simulate a wartime environment like Red Flag 
are also considered as “combat flights.” 

PERSONAL CHRACTERISTICS 

1. What is your current rank? 

1. Second Lieutenant 

2. First Lieutenant 

3. Captain 

4. Major 

5. Lieutenant Colonel 

6. Colonel 

2. What is your gender? 

1. Female 

2. Male 

3. How many hours of flight experience do you have, excluding flight 
training ? 

1. less than 500 hrs. 

2. 501-1000 hrs 

3. 1001-1500 hrs 

4. 1501-2000 hrs 

5. 2001-2500 hrs 

6. 2501-3000 hrs 

7. over 3000 hrs x.hrs (specify) 

4. What is the current aircraft you are flying or the most recent aircraft 

you flew? X.(specify) 
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5. What other aircraJfit have you flown? (Please list in reverse 
chronological order—^Most recent first, and so on.) 


X.(specify) 

X.(specify) 

X.(specify) 


6. How many hours of flight experience do you have in combat 
(officially logged combat hours)? 

1. none 

2. less than 100 hours 

3. 101-200 hours 

4. 201-300 hours 

5. 301-400 hours 

6. 401-500 hours 

7. more than 500 hour 

X.(specify) 

7. What type(s) of aircraft did you fly in combat? (Please see the 
definition of combat above, and list them in reverse chronological 
order, if more than one.) 


X.(specify) 

X.(specify) 


8. What primary mission did you fly in combat? (Please specify the 
most recent one if you participated in combat more than once.) 

1. Strategic Deterrence 

2. Conventional Heavy Bombing Operations 

3. Theater Airlift 

4. Tactical Airlift 

5. Air-to-Air Refueling 

6. Air Superiority 

7. Close Air Support 

8. Air Interdiction 

9. Combat Air Patrol 

10. Other x.(specify). 
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WORKLOAD FACTORS 


For the rest of the questions presented in this questionnaire, please answer them 
according to the aircraft you specified for Question 4. 

For questions 9-37, please evaluate each of the following factors in terms of the degree to 
which you believe each is likely to increase the pilot workload in a combat environment. 
Please assign a value between 1 to 6 according to the Likert scale provided. In this scale: 

“Dangerous increase” means any increment in workload that might cause extreme 
delays in the task currently being performed or unsafe situations or mission failure. 

“Distracting increase” means any increment in workload that might cause excessive 
delays in the task currently performed, or recoverable xmsafe situation or decrease in 
mission performance. 

“Moderate increase” means any increment in workload that might cause moderate 
delays in the task currently being performed. 

“Some increase’ means any increment in workload that might cause some delays in 
the task currently being performed. 

“Little or No increase” means any increment in workload that might cause little or no 
delay in the task currently being performed. 


1. Dangerous 2. Distracting 3. Moderate 4. Some 5. Little or No 6. Not 

Increase Increase Increase Increase Increase Applicable 


9. Mission Planning 1 

10. Terrain Avoidance/Following 1 

11. Maintaining Situational 1 

Awareness 

12. Adverse Weather 1 

13. Monitoring Flight Instruments 1 

14. Equipment Degradation 1 

15. Low Level Navigation 1 


2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 

2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 
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For questions 16-30, please evaluate each of the following factors in terms of the degree 
to which you believe each is likely to increase the pilot workload in a combat 
enviromnent? 


1. Dangerous 2. Distracting 3. Moderate 4. Some 5. Little or No 6. Not 

Increase Increase Increase Increase Increase Applicable 


16. Night Low Level Navigation 1 

17. Threat Avoidance 1 

18. Formation Responsibilities 1 

19. Management of TOT 1 

20. In-flight, No-Notice Mission 1 

Changes 

21. Shifting Attention to 1 

Targets of Opportunities 

22. Munitions Deployment 1 

23. Threat Detection 1 

24. Crew Incapacitation 1 

25. In-flight Emergency 1 

26. Visual Orientation 1 

27. Command & Control 1 

(such as copying&decoding EAMS) 

28. Fatigue 1 

29. Crew Coordination 1 

30. Aircraft Maneuvering 1 

(Dogfight or avoiding the threats) 


2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 

2 3 4 5 6 

2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 

2 3 4 5 6 
2 3 4 5 6 
2 3 4 5 6 
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For questions 31-37, please evaluate each of the following factors in terms of 
the degree to which you believe each is likely to increase the pilot workload in 
a combat environment? 


1. Dangerous 2. Distracting 3. Moderate 4. Some 5. Little or No 6. Not 

Increase Increase Increase Increase Increase Applicable 


31. Target Acquisition 

32. Type of Drop 

33. Night Operations 

34. Unfamiliar Terrain 

35. Managing Radio Communication 

36. Refueling Operations 


1 2 3 4 5 6 
1 2 3 4 5 6 
1 2 3 4 5 6 
1 2 3 4 5 6 
1 2 3 4 5 6 
1 2 3 4 5 6 
1 2 3 4 5 6 


37. Responding to Groxmd/Airbome 
Controller instructions. 

37a. Do you have any thing to add to the list above? (Please write your 
answer below, or skip the question if your answer is no.) 


For Questions 38-44, please evaluate each statement in terms of the degree that 
you agree with each statement by using the following scale. 


1. Strongly 2. Agree 3. Neutral 4. Disagree 5. Strongly 

agree Disagree 


38. Additional aircrew could reduce the workload in a combat environment 

(Assuming that your aircraft could be designed in such a way to accommodate the 
additional aircrew such as Weapon Systems Officer (WSO), Electronic Warfare 
Officer (EWO), Navigator, or Radar Intercept Officer (RIO). 

1 2 3 4 5 
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For Questions 39-44, please evaluate each statement in terms of the degree that you agree 
with each statement by using the following scale. 


1. Strongly 
agree 


2. Agree 3. Neutral 4. Disagree 5. Strongly 

Disagree 


39. Modifying the cockpit resources of your aircraft could help to eliminate the excess 
workload in the cockpit of the aircraft you flew in combat. 

1 2 3 4 5 

40. Technological innovations in future aircraft designs will help reduce the 
workload. 


1 2 3 4 5 

41. Updating the current operational regulations and procedures could help eliminate 
excess workload in the cockpit of the aircraft you fly or the most recent aircraft 
you flew. 


1 2 3 4 5 

42. The workload of combat flight operations is heavier than that of peace time 
operations. 


1 2 3 4 5 

43. Simulator missions flown in peace time adequately simulate the amount of 
workload that a pilot will experience in combat. 

1 2 3 4 5 

44. In combat, the superiority of the U.S. and allies’ air power over that of enemies 
would have an positive impact in reducing the amount of workload experienced 
during in-flight operations. 

1 2 3 4 5 


A-8 



44a. During peace time operations (or training missions), what scenario or procedure 
did you think would be most difficult to handle in combat (such as being detected 
by the enemy defense, or having a emergency in the target area)? (Please write 
your answer below.) 


45. Did you find it (Question 44a scenario) difficult to handle in the combat 
environment? 

1. Yes, even harder than I had expected. 

2. No, it was about the same. 

3. It was less. 

4. Not applicable 

45a. If your answer to the Question 45 is “It was less,” was there any procedure that 
you found more difficult to handle than you expected? (Please write your 
answer below, or skip if not applicable.) 


46. Would you volunteer to participate in experimental studies which measure 
workload using data collected in flight in a combat environment? 

1. Yes 2. No. 

46a. If your answer to Question 46 is “No,” please write your reason. 


47. Would you volimteer to participate in experimental studies mentioned above in a 
simulated combat mission? 

1. Yes 2. No. 
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47a. If your answer to Question 47 is “No,” please write your reason. 


48. Please at this moment feel free to make any comment about the issues discussed 
in the survey itself. You may xise the available space on this page or attach extra 
paper if you need. 
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Appendix B: Survey Comments 


Introduction 

The final question, Question 48, on the survey questionnaire invited pilots to 
make any comments regarding the issues discussed in the survey. There was no attempt 
made to guide respondents to provide a particular information or to discuss a specific 
issue. The comments were made to clarify or to detail the opinions of pilots who elected 
to express their concerns about pilot workload and combat. Comments on Questions 46a, 
and 47a, were included in the appendix, if respondents elected to answer the questions. 
These questions asked pilots to provide reasoning for not preferring to volxmteer 
workload assessment experiments in combat missions and simulators, respectively. The 
conunents on the structure and contents of the survey were exempted from the appendix. 
Overall, the comments of pilots with combat experience should provide valuable insights 
into combat workload assessment. The conunents are also useful for making qualitative 
judgments on the statistical data. Below, listed are the pilots’ comments from six aircraft 
surveyed concerning Questions 46a, 47a, and 48. 
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AO/A-10 


Ten pilots elected to include their comments. 

Question 48: Workload in combat and the ability to deal with it is a direct result 
of training! We need to train to go to combat! Simulators do not do it. We need 
to fly the missions on tactical ranges, employ weapons, and learn from our 
mistakes. Bottom line: experience is the answer that you are looking for. 


2. Major, 2001-2500 total flying hours, less than 100 combat flying hours: 

Question 48: We, the Air Force members, need to quit focusing our technological 
energies or money on fighting the last war (Desert Storm); and we should think 
towards future conflicts which will be low intensity and permissive threat 
environments. All the cockpit magic in the world will not be helpful if the pilot 
and/or the groimd commander does not know where the friendlies are. The 
EPLARs-SADL system is a good start. Identifying targets and friendlies in an 
urban, jungle or moimtain environment with no clear lines of demarcation and 
merged forces is where your focus should be. 


3. Captain, 2001-2500 total flying hours, 301-400 combat flying hours: 

Question 48: A-10 needs a reliable, fully navigation-modified cockpit and GPS. 


4. Captain, 2501-3000 total flying hours, 210-300 combat flying hours: 

Question 46a: Combat is not the time for studies and experimentation; however, 
most of what is being logged as “combat flights” these days (Southem/Northem 
Watch, Bosnia) is not really combat anyway. So, it might be appropriate. 

Question 48: Any modifications to aircraft that make the cockpit more “pilot 
fiiendly,” (i.e. better ergonomics, switch positions, etc.) will be beneficial for 
combat operations. Also, the more automated the weapon delivery system is, the 
easier it will be for the pilots to concentrate on target acquisition and staying 
alive. 
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5. Major, 2001 -2500 total flying hours, less than 100 combat flying hours: 

Question 48: Desert Storm was not the best test for a high threat war. Some 
combat pilots flew definite high threat missions; but, many were low threat 
tactics. Medium-altitude tactics are very different in high-threat or medixun-threat 
areas. It should be kept in mind while assessing workload. 


6. Colonel, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 46a: I have participated in workload assessment studies before. The 
results are pre-determined; and, the goal of the testing is to achieve those results. 
In any case, the results are meaningless unless they result in something removed 
from the cockpit before anything may be added. 


7. Major, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 46a: I would have enough to worry about in combat without adding 
academic studies to my workload. I do not want anything unfamiliar to me that I 
can control in the cockpit in a combat environment. 

Question 48: As new systems are contemplated to make air and space power 
more, we must always remember that combat is very complex; and, so are the air 
operations. The fact that we may have “made it look easy” should not fool us into 
thinking it is. I get very nervous when folks talking about re-targeting a mission 
after the crews have thoroughly planned to do something else. We cannot think of 
air power in the same way as we think of artillery—just plug in new coordinates, 
move the barrel, and pull the lanyard. Air operations are not that simple. Routes 
have to be plaimed and de-conflicted based on a number of factors. Some of these 
are threats, weather, tankers, EW support, ground and naval operations, fuel 
required, avenues of approach, target type, target acquisition, perhaps moon 
illumination, elevation and etc. These are all factored into planning each mission 
and thoroughly briefed. To expect crews to able to just input new coordinates, 
and “using it,” especially for interdiction, counter-air and strategic attack 
operations is to unnecessarily risk the lives of the crew and the aircraft. Expect 
increased losses, and decreased mission effectiveness. Certainly, CAS and some 
air superiority missions lend themselves more to flexibility. However, in most 
cases even CAS missions are re-targeted within the a general area (not always 
though). 
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Crews still need time to plan, come up with backups, and study the target(s) 
against which they are being asked to risk their lives whenever possible. The 
ability to rapidly re-target is appealing and necessary; but, those making decisions 
in the air operations centers must always weigh the added risk you expose your 
force to when you send it in with less than thorough preparation. 

Emphasis needs to be on getting needed intelligence to the aircrew (preferably for 
mission planning). Also, the emphasis should be placed on systems like data 
modems to automatically load targets, IP, and checkpoints, into navigation 
systems in flight. Target acquisition aids in greater standoff and resolution are 
needed. The need for identifying friends or enemies positively, the need for better 
and automatic radar and IR threat countermeasure systems, and the need for 
improved situation displays that incorporate near-real time data like Joint STARS, 
AWACS, should also be emphasized. 


8. Captain, 1001-1500 total flying hours, less than 100 combat flying hours: 

Question 48: One item which affected me a lot by loading on more stress and 
fatigue was the twice nightly attacks which interrupted my sleep patterns. We 
would be awakened by sirens at approximately 2200 and 0200 hours every night. 
Each night we had to put on full chemical gear and jump into bunkers for about an 
hour at a time. Then, we would still get up at dawn, and fly two to three sorties of 
combat missions. 


9. Captain, 2001-2500 total flying hours, 201-300 combat flying hours: 

Question 48: A true combat environment is tough to simulate. We train hard to 
a combat environment, but until the shooting starts, and the possibility of death 
exists, you can never gather accurate information. Emotions run high in combat 
and some people perform better in that environment then others. I have trained 
with some of the best, but in a combat environment personalities change. 


10. Captain, 1501-2000 total flying hours, less than 100 combat flying hours: 

Question 48: One of the biggest problems is communications with multiple 
groimd agencies. So many users on the same frequency tend to jam the 
communication on the frequency. Also, the layout of the radios in the A-10 
causes excessive “hands-down” time when switching frequencies. 
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B-52 


Ten B-52 pilots elected to write comments. 

1. Major, 2501-3000 total flying hours, 201-300 combat flying hours: 

Question 48:1 found combat flying much easier than training. Reason: many of 
peacetime micromanagement rules went away, and real goal of fly, fight, and win 
came to the surface. 

2. Major, 2501-3000 total flying hours, 201-300 combat flying hours: 

Question 46a: Why would I want to make combat more challenging by trying to 
study it—^added tasks with dubious valve? I do not think that I would prefer that. 

Question 47a: Because, you cannot simulate the stress of life and limits without 
actually risking them—^the study will produce fatally flawed outcomes. 

3. Major, 3100 total flying hours, 101-200 combat flying hours: 

Question 48: The B-52 combat mission tasks are far easier at high altitude than 
they are at low altitude such as Air Superiority, Terrain Avoidance, and Weapon 
Delivery, etc. My answer related to the worst case scenario, my two low level 
combat sorties. That may be a moot point, especially if the buff will never go low 
again. 


4. Captain, 1001-1500 total flying hours, no combat flying hours: 

Question 48: Communication, internal and external, is a major player in 
increasing workload. We have 3 UHFWHF radios, IHF, secme voice, have- 
quick, SATCOM, and more. It is tough to monitor too many sources while 
attempting to attack a target. 


5. Major, 3100 total flying hours, 101-200 combat flying hours: 

Question 47a: Simulators cannot match the required stress loads on pilots. 
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6. Major, over 3000 total flying hours, 101-200 combat flying hours: 

Question 48: Simulators are still very limited in raising pucker factor. There is no 
threat of death, aircraft response is not quite the same, and “fog of war” difficult 
to evaluate. 


7. Captain, 2001-2500 total flying hours, 101-200 combat flying hours: 

Question 48:1 believe this study is well worth the effort. Given the correct state 
of personnel management, I do not believe there are still a significant number of 
active duty flyers with combat experience. That loss of experience base could be 
very costly to regain. 


8. Colonel, 2001-2500 total flying hours, less than 100 combat flying hours: 


Question 47a: You cannot truly re-create a combat mission. Training rules cannot, 
and should not allow it. 

Question 48: Despite the success of Desert Storm, we all know that a war is not 
won in the skies, but on the ground. As glamorous as air combat may seem to be, 
we are an extension of ground combat. We need to keep that in mind. 


9. Captain, 2501-3000 total flying hours, 101-200 combat flying hours: 

Question 48: It takes more than new airframes to solve pilot concerns and combat 
workloads. Many (systems) technologies which work, are available, and could be 
pun on older aircraft to reduce workload and greatly increase threat penetration. 
Developing new aircraft and capabilities is a given, but upgrading current systems 
must be given top priority. If an aircraft cannot pull 10 G’s or needs only a small 
amount of funding to upgrade its current ECM system, DO IT! You can keep my 
pilot bonus if you have to to do it!! Recce and transports need ECM upgrades. 

Just like the fighters and bombers. 


10. Major, 3500 total flying hours, 101-200 combat flying hours: 

Question 48: Our preparation for combat is the best it can be. Desert Shield 
provided the opportunity to work directly with AWACS, as well as heavy-weight 
night refueling operations behind KC-lOs. Combat will always be unpredictable, 
and preparing for it extremely difficult. However, simulators along with exercises 
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(Desert Shield) will keep us on the leading edge. It is paramount that we judge 
performance rather than potential. 


C-130 


Eleven C-130 pilots elected to include comment. 

1. Major, 4300 total flying hours, less than 100 combat flying hours: 

Question 48: Another aspect to address is the participation of units/personnel in 
the humanitarian and Civil Military Operations that the U.S. has been increasingly 
involved in. Although not “combat,” the small arms threat and groimd threat 
exists in the environment crews are flying in, and in the airfield /dirt strip 
environment the support troops live in. 


2. Captain, 1001-1500 total flying hoius, less than 100 combat flying hours: 

Question 48: This survey should separate combat training to include Red Flag, 
from the real thing. We train a lot harder than we fight. Supporting Joint 
Endeavor and Southern Watch was much easier (less workload) than Red Flag or 
JRTC (a C-130 combat exercise). It was hard to generalize all combat 
environments to answer the questions, because each operation, or exercise is very 
unique to itself. Be careful of replacing bodies with technological advances. The 
C-130J is beginning to do this, and I am sme the workload for the pilots will 
increase. The navigator is integral to the success of the mission. 


3. Major, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 48:1 am just a C-130 pilot with limited time going into Bosnia. All of 
my crews’ threat reactions were due to false alarms. I am sure the pucker factor, 
and alertness, would have gone up if a missile were to have gone whizzing by. 


4. Captain, 1501-2000 total flying hours, less than 100 combat flying hours: 

Question 48: Technological updates to the older Hercules would be a huge help 
in advancing situational awareness and reducing workload. Many projects have 
been identified, but not funded. Fimd them and the Hercules world will become 
much safer. 
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5. Major, 3850 total flying hours, less than 100 combat flying hours: 

Question 48: Daily battle plan issued to C-130s was very unwieldly. Not 
workable for emergency plans; too detailed without a good overall concept. 

6. Major, 4500 total flying hours, less than 100 combat flying hours: 

Question 48: The C-130 Simulator Refresher Course at Little Rock AFB includes 
a scenario called the Mission Oriented Simulation Training (MOST). It is a 
simulated combat mission designed to assess Cockpit Resource Management 
(CRM) techniques and skills. It provides excellent training, and sounds very 
much like what you allude to in Questions 46, and 47. 


7. Major, 3700 total flying hours, less than 100 combat flying homs: 

Question 48: Good, sound crew coordination and management can make or break 
the mission. Aircraft commanders must be familiar with the capabilities and 
limitations of his aircraft and crew. Knowing this can keep the options open in 
the changing combat environment. Time spent with a hard crew (the same crew 
members) increases this knowledge. I know that this is a broad statement, but it is 
a key to knowing how flexible you can be as situations change, and they will 
change in combat. 


8. Captain, 2501-3000 total flying hours, 101-200 combat flying hours: 

Question 48: Reducing the number of crew members in the C-130 cockpit (even 
with technological advances) will at best cut the C-130 combat capability in half, 
and at worst, make it impossible or much too dangerous. 


9. Captain, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 48: Although I volunteered to participate in an experimental study, it 
would not be a fair judgment of overall crew abilities. I am rather experienced. 
You would need to get young, “line” crews for reasonable results. 
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10. Captain, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 46a: I do not want to any additional distractions in a combat flight 
environment. 

11. Captain, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 48: Our regulations are currently being updated and changed to Air 
Force Instructions (APIs). Additionally, there have been so many changes to the 
operational guidance, crews cannot keep their publications updated 


F-15 


Twelve F-15 pilots included survey comments 

1. Major, 2501-3000 total flying hours, 201-300 combat flying hours: 

Question 48:1 found that practicing tactics on training missions for seven years, 
allowed me to form habits which during the heat of battle caused me to react 
instinctively. (Habitual behavior) The bottom line is we need to practice. Flying 
and simulators; however, more in the aircraft. 


2. Captain, 501-1000 total flying hours, less than combat flying hours: 

Question 48: Twin engine, two seat fighters like the F-15E have a vast advantage 
in high threat missions. Tasks can be divided and full attention paid to a number 
of different tasks. Having the second engine to use as required immeasurably 
lowers the risk and stress level. 


3. Major, 2501 -3000 total flying hours, less than combat flying hours: 

Question 46a: A combat sortie is inappropriate for testing. Where would the 
emphasis be—data collection or bombs on target? Would aircrews be required to 
perform nonstandard maneuvers, or fly in test modified aircraft? If the data 
collection was completely transparent, it might be acceptable. 
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4. Major, 2501-3000 total flying hours, 101-200 combat flying hours: 

Question 46a: The only true combat environment is a real one, and there have 
never been two that are same. Plus, I choose not to go to combat unless required, 
and do not plan to be part of an experiment during that time. 


5. Major, 2501-3000 total flying hours, 101-200 combat flying hours: 

Question 48:Important elements missing from the combat workload items’ 
listings are Commimication Jamming or Degradation, Working as a Commander 
or Flight Lead in Large Composite Scenarios. Item 12, Adverse Weather: I 
interpreted to be the worst scenarios: target weather which drives me down into 
more lethal A3/MANPADs threat system(s); other weather issues include enroute 
and aerial refueling severe weather. Lastly, your survey did not mention the stress 
of operations in an NBC environment—even Saudi Arabia was a CW AOR! 


6. Major, 2501-3000 total flying hours, 101-200 combat flying hours: 

Question 48: All the neat gee-whiz stuff in current fighters creates information 
overload, not less workload. 


7. Major, 2001-2500 total flying hours, less than 100 combat flying hours: 

Question 48: Training is the key for workload management in combat. When I fly 
missions I have recently trained in, the workload is more manageable. 

Technology can help, and hurt your workload depending on the way it is applied. 
Technology without considering human factors in not good. Recent F-15 
improvements which help eliminate workload include the MFD Situation Display 
and JTIDs. Although my unit did not have JTIDs, I have used it in the simulator. 
JTIDs is incredibly helpful and workload reducing for the air-to-air environment. 


8. Major, 2001-2500 total flying hours, less than 100 combat flying hours: 

Question 48: Combat is too restrictive a term. I logged combat time during 
Operation Southern Watch. Much different environment than 0300 hours on 17 
January 1991. 
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9. Major, 3400 total flying hours, 201-300 combat flying hours: 

Question 48: Right now, the average fighter pilot’s skills are less than when we 
fought the Desert Storm. Too many non-tactical TDYs like Southern Watch, 
Deny Flight, Roving Sands, etc. Last year, we could not go to Red / Green Flag, 
because of all the otitier bs.. .tasking. Skills after a FLAG exercise are always 
much higher. Doing more with less is taking its toll. 


10. Major, 2501-3000 total flying hours, 301-400 combat flying hours: 

Question 46a: No experiment is required. Come and talk to the crews. If you set 
up an experiment and not know what you are doing, the results will be too 
expensive and poorly aimed. 

11. LTC, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 48: Procedures have to be ingrained—second nature—during training 
sorties to prepare for combat. Pilots who attempt to learn their jet or do 
something off the shelf during a combat sortie tend not to get the desired results. 
Serious training is required. 

12. Major, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 48: You are wasting your time; however, in the above experiments, we 
practice harder scenarios than we actually see in combat. We have no way to 
simulate in peace, or the additional stress of knowing that death will be very real 
instead of an administrative kill—removal. 


F-16 


Eleven F-16 pilots wrote comments. 

1. LTC, 2501-3000 total flying hours, 101-200 flying hours: 

Question 48: The United States Air Force’s technological advances, and tactical 
and operational improvements continue to provide qxiantum leaps in our ability 
project aerospace power. 
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2. LTC, 3100 total flying hours, 101-200 combat flying hours: 

Question 48:Training, lots of it, is the best way to help a pilot feel confident in his 
or her abilities, systems, and aircraft. Experience will better prepare someone to 
deal with the dynamics of the combat environment. 


3. Captain, 1501-2000 total flying hours, less than 100 combat flying hours: 

Question 47a: Simulations, well, are just that, and I found it hard to believe that 
you could ever get over the fact you always at “1-G” flight. 

Question 48: We are hurting in the operations world just to get enough flights to 
stay current in all requirements, and do not get many opportunities to train in 
large force exercises, simulating combat. We deployed to Southwest Asia; flying 
there is so tame and over controlled. It detracts from our overall combat 
capability, therefore our stress level, if it actually happened. 


4. Captain, 1001-1500 total flying hours, 101-200 combat flying hours: 

Question 48: There are a lot of “off-the-shelf’ technologies that could reduce 
workload especially in combat environment. Onboard identification systems, 
moving maps, GPS, Datalink that are being delayed for political reasons. They 
could be the difference between surviving or perishing in a combat environment. 


5. Major, 1501-2000 total flying hours, 101-200 combat flying hours: 

Question 48: First, your survey is too long which will affect the accuracy of your 
results—^Human Reliability. Second, a horizontal situation display will reduce 
mental workload and free the resources to make tactical decisions. Also, we 
develop training scenarios that are tougher than many of the real world scenarios. 


6. Major, 1501-2000 total flying hours, less than 100 combat flying hours: 
Question 46a: There should be no testing in combat. 
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Question 47a: No, I do not have time for one more TDY. I have already 
participated in a study that tries to increase the speed of a simulator (1.5 or 2X the 
normal speed) to help control things in air better. 


7. Captain, 1501-2000 total flying hours, 301-400 combat flying hours: 

Question 48: “Real” combat versus contingency operations, that is Provide 
Comfort, is completely different as is a high air threat scenario, or a high-tech 
surface threat scenario in terms of vv^orkload and its effects on safety and mission 
accomplishment. 


8. LTC, 3100 total flying hours, less than 100 combat flying hours: 

Question 48: The biggest concern I ctirrently have about the Air Force preparing 
for combat is that we are training far less than the way we are expected to fight, 
due primarily to fiscal constraints, and the lack of spare parts. For monthly 
“bookkeeping” procedures, however, the reporting procedures have been changed 
to make it “appear” that the combat air forces are strong, when, in fact, they are 
not. I would not feel comfortable as a yotmg pilot just starting out as opposed to 
being a seasoned pilot on the eve of my career, and able to retire in the near 
future. I strongly feel that these reasons are a major reason why pilot retention is 
becoming so severe. 


9. Major, 2501-3000 total flying hours, 101-200 combat flying hours: 

Question 47a: A simulator is not the real thing as much as you try and make it so. 
My life and liberty are not on the line in the simulator. 


10. Major, 1501-2000 total flying hours, less than 100 combat flying hours: 

Question 48: The restrictions that we operate imder in peace time do not allow us 
to adeqizately prepare for combat. For example, we caimot go supersonic in the 
Military Operating Air Spaces (MO AS) we fly in. This means much of our air 
intercept training is unrealistic. Also, we are restricted from deploying chaff and 
flare in virtually every area we fly; this is also unrealistic, and hinders our 
training. 
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11. Captain, 501-1000 total flying hours, 101-200 combat flying hours: 

Question 48: As the F-16C gains more and more combat missions—^INT, DCA, 
FAC(A), SEAD, and CAS (I do not think that there are any other missions to give 
us.), pilots rely more on technology and aircraft ergonomics to make their jobs 
possible. Here is what we need: 

1. NVGs 

2. IFF interrogator 

3. HSD 


KC-135 


Seven KC-135 crew members returned comments. 

1. Captain, 2001-2500 total flying hours, less than 100 combat flying hours: 
Question 48: We need RAW gear in the tanker aircraft. 


2. Major, 2501-3000 total flying hours, less than 100 combat flying hours: 

Question 48: For a KC-135, our job is pretty much the same in peace time and 
combat. The problems arise when it comes to threat detection. Currently, the 
KC-135 has no way of knowing of enemy aircraft location, or a missile launch. 
We depend heavily on AWACS for this type of information. 


3. Major, 3200 total flying hours, 101 -200 combat flying hours: 

Question 46a: A combat environment is not the right place for experimental 
studies. 


4. Captain, 1001-1500 total flying hours, less than 100 combat flying hours: 

Question 48: KC-135 operations have moved away from “01”missions and 
possible environments, which is a good thing. Because, we have limited detect 
capability (if any), and no defense capability (chaff, flare, etc.) 
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5. Major, 3500 total flying hours, 201-300 combat flying hours: 


Question 48: This survey was directly aimed at fighter/bomber aircrews. Nothing 
wrong with this, but why survey tanker crew members. If the air force were 
serious about this for tankers, they would not be eliminating the navigator 
positions in the KC-135. 


6. LTC, 3600 total flying hours, less than 100 combat flying hours: 

Question 48: The HQ staff at AMC and NAF staff are not willing to change the 
methods used in flight by the KC-135. Even the leadership addresses change, but 
the Air Force Instructions and Technical Orders do not address, or adapt to new 
ways of doing the job. 


7. Captain, 1501-2000 total flying hours, 101-200 combat flying hours: 

Question 48: In the tanker world, we do the same peace-time missions as we 
would do in wartime missions, except for large formation refuelings—^that is 4,5, 
or 6 ship formations. We did it everyday in the war, and in the aftermath of the 
war, but we stopped doing it years ago. So the skills and workload of pilots 
increase dramatically in very large formations. 
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Appendix C: SAS Computer Program 


options linesize=80; 

filename surveys 'surveyz.dat'; 

data temp; 

infile surveys missover; 
input @41 {Q1-Q48)(48*1.); 

If Q8=0 then Q8=.;If Q8=l then Q8=,;If Q5=. then Q5=0; 

If (Q4=l or Q4=4 or Q4=5) and Q3>2 and Q6>2 and Q5>0 then Q48=l; 

If (Q4=2 or Q4=3 or Q4-6) and Q3>4 and Q6>3 and Q5>0 then Q48=l; 

If Q48 ne 1 then Q48=0; 

If Q9=6 then Q9=.;If Q10=6 then Q10=.;If Qll=6 then Qll-.; 

If Q12=6 then Q12=.;If Q13=6 then Q13=.;If Q14=6 then Q14=.; 

If Q15=6 then Q15=.;If Q16=6 then Q16-.;If Q17=6 then Q17=.; 

If Q18=6 then Q18=.;If Q19=6 then Q19=.;If Q20=6 then Q20=.; 

If Q21=6 then Q21=.;If Q22=6 then Q22-.;If Q23=6 then Q23=.; 

If Q24=6 then Q24=.;If Q25=6 then Q25=.;If Q26=6 then Q26=.; 

If Q27=6 then Q27=.;If Q28=6 then Q28=.;If Q29=6 then Q29=.; 

If Q30=6 then Q30=.;If Q31=6 then Q31=.;If Q32=6 then Q32=.; 

If Q33=6 then Q33=.;If Q34=6 then Q34=.;If Q35=6 then Q35-,; 

If Q36=6 then Q36=.;If Q37=6 then Q37=.; 

CWL=sum(Q10,Qll,Q12,Q14,Q15,Q16,Q17,Q18,Q19,Q20,Q21,Q22,Q23, 
Q24,Q25,Q26,Q27,Q28,Q29,Q30,Q31,Q32,Q33,Q34,Q35,Q36,Q37); 

Proc Format; 

Value Acft 1='A-10’ 

2-’B-52* 


Value Mission 


Value Exprnce 
Value TotFltHr 


3='C-130’ 

4=’F-15’ 

5='F-16» 

6=’KC-135’; 

1=’Strategic Deterrence’ 
2=’Conv. Hvy.Bmbg Ops’ 

3-’Theater Airlift’ 

4=’Tactical Airlift’ 

5=’A/A Refling’ 

6=’Air Superiority’ 

7=’CAS’ 

8=’Air Interdiction’ 

9=’CAP’ 

0=’Others’; 

1=’Experienced Pilot’ 

0=’Inexperienced Pilot’; 
1=’< 500 Mrs’ 

2- '501-l,000 Hrs’ 

3- ’l,001-1,500 Hrs’ 

4='l,501-2,000 Hrs’ 

5=’2,001-2,500 Hrs' 

6=’2,501-3,000 Hrs' 

7=’> 3,000 Hrs’; 
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Value ComFltHr l=*None* 

2-’< 100 Hrs» 

3=’101-200 Hrs’ 

4-*201-300 Hrs' 

5=’301-400 Hrs’ 

6=M00-500 Hrs’ 

7=’> 500 Hrs’; 

Value Rank l=’2nd Lt’ 

2=’1st Lt’ 

3=’Captain’ 

4-’Major’ 

5=’Lt Col’ 

6=’Colonel’; 

Value Gender 1=’Female’ 

2=’Male’; 

Proc Sort data=temp out=SortAC; 

Format Q4 Acft.; 

by Q4; 

run; 

Proc Sort data=temp out=Missn; 

Format Q8 Mission.; 

by Q8; 

run; 

Proc Sort data=temp out=exper; 

Format Q48 Exprnce.; 
by Q48; 
run; 

Proc Print Data=SortAC; 

Format Q4 Acft.; 

Title ’Print-out of Entire Data with Aircraft Type’ 
run; 

Proc Freq Data=SortAC; 

Format Q1 Rank.; 

Format Q2 Gender.; 

Format Q3 TotFltHr.; 

Format Q6 ComFltHr.; 

Format Q8 Mission.; 

Format Q4 Acft.; 

Format Q48 Exprnce.; 

Title ’Frequency Print-out of Entire Data’; 
run; 

Proc Freq Data=SortAC; 

Format Q1 Rank.; 

Format Q2 Gender.; 

Format Q3 TotFltHr.; 

Format Q6 ComFltHr.; 

Format Q8 Mission.; 

Format Q4 Acft.; 

Format Q4 8 Exprnce.; 

Title ’Frequency Print-out by Aircraft Type’; 
by Q4; 
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run; 


Proc Means Data=SortAC; 

Var Q9 QIO Qll Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 
Q23 Q24 Q25 Q26 Q27 Q28 Q29 Q30 Q31 Q32 Q33 Q34 Q35 Q36 Q37; 

Title ’Means of the Entire Critical Workload Factors’; 

Proc Means Data=SortAC; 

Format Q4 Acft.; 

Var Q9 QIO Qll Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 

Q23 Q24 Q25 Q26 Q27 Q28 Q29 Q30 Q31 Q32 Q33 Q34 Q35 Q36 Q37; 

by Q4; 

Title ’Means of Critical Workload Factors Sorted by Aircraft 

Type’; 

run; 

Var Q9 QIO Qll Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 

Q23 Q24 Q25 Q26 Q27 Q28 Q29 Q30 Q31 Q32 Q33 Q34 Q35 Q36 Q37; 

by Q48; 

Title ’Means of the Entire Critical Workload Factors of all pilots 
by Experience’; 
run; 

Proc Factor Data=SortAC simple nfact=2 reorder; 

Var Q9 QIO Qll Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 Q23 

Q24 Q25 Q26 Q27 Q28 Q29 Q30 Q31 Q32 Q33 Q34 Q35 Q36 Q37 Q38 Q39 

Q40 Q41 Q42 Q43 Q44 Q45 Q46 Q47; 

Title ’Factor Analysis of Combat Workload Factors’; 
run; 

Proc Corr Alpha Nocorr; 

Var QIO Qll Q12 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 Q23 

Q24 Q25 Q26 Q27 Q28 Q29 Q30 Q31 Q32 Q33 Q34 Q35 Q36 Q37; 

Title ’Reliability Test of Combat Workload Factors’; 

Run; 

Proc Reg Data=temp; 

Model CWL=Q1 Q3 Q4 Q5 Q6 Q48 Q8/Selection=Stepwise AdjRsq; 

Title ’Stepwise Reg. of CWL —Entire Sample’; 

Run; 

Proc GLM Data=SortAC; 

Format Q4 Acft.; 

Class Q4; 

Model CWL=Q4 Q1 Q8 Q48 Q4*Q1 Q4*Q8 Q4*Q48 Q1*Q8 Q1^Q48 Q8*Q48; 
Means Q4/LSD BON SCHEFFE; 

Title ’Means Comparison of CWL by Aircraft Type’; 

Run; 

Proc Reg Data=SortAC; 

Model CWL=Q1 Q3 Q5 Q6 Q48 Q8/Selection=Stepwise AdjRsq; 

By Q4; 

Title ’Stepwise Reg. of CWL—By Aircraft Type’; 

Run; 

Proc Anova; 

Class Q4; 

Model CWL=Q4; 

Means Q4/LSD; 

Title ’Anova Procedure of Aircraft’; 
run; 

Proc Anova; 

Class Q6; 

Model CWL=Q6; 

Means Q6/LSD; 

Title ’Anova Procedure of Combat Flight Time’; 
run; 


C-3 



Proc Anova; 

Class Q48; 

Model CWL=Q48; 

Means Q48/LSD; 

Title ’Anova Procedure of Experience*; 
run; 

Proc Anova; 

Class Q8; 

Model CWL=Q8; 

Means Q8/LSD; 

Title ’Anova Procedure of Mission Type’; 
run; 

END SAS; 
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A ppendix D: Combat Workload Items’ Ratings 


Combat Workload Items bv All Pilots; 


Table D-1 


Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. Threat Avoidance 

2.2146 

0.9358 

1.0000 


2. In-flight Emergency 


0.9348 

1.0000 




Table D-2 

Combat Workload Items Found to be Moderately Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Fatigue 

2.6303 

1.1150 

1.0000 

6.0000 

2. 

Crew Incapacitation 

2.6303 

1.9169 

1.0000 

6.0000 

3. 

Adverse Weather 

2.6575 

1.0390 

1.0000 

6.0000 

4. 

Aircraft Maneuvering 
(dogfight or avoiding threats) 

2.7763 

1.2922 

1.0000 

6.0000 

5. 

In-flight, No-Notice 

Mission Changes 

2.8493 

0.9138 

1.0000 

6.0000 

6. 

7. 

Equipment Degradation 
Maintaining Situation 

6.0000 

Awareness 

2.9726 

0.9857 

3.3288 1.1380 

1.0000 

1.0000 

6.0000 

8. 

Threat Detection 

3.3379 

1.1982 

1.0000 

6.0000 

9. 

Night Operations 

3.3470 

1.0310 

1.0000 

6.0000 

10. 

Night Low Level 

Navigation 

3.3562 

1.6539 

1.0000 

6.0000 

11. 

Terrain Avoidance/ 

Terrain Following 

3.4658 

1.3722 

1.0000 

6.0000 
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Table D-3 


Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 


Std. Dev. 

Minimum 

Maximum 

1. 

Unfamiliar Terrain 

3.6073 


1.0968 


1.0000 


6.0000 

2. 

Command and Control 


3.6101 


1.1558 


1.0000 



6.0000 









(such as copying & 
decoding EAMS) 








3. 

Low Level Navigation 


3.7397 


1.2889 


1.0000 



6.0000 








4. 

Managing Radio 
Communication 

3.7580 


0.9534 


2.0000 


6.0000 

5. 

Formation 

Responsibilities 

3.7854 


0.9742 


2.0000 


6.0000 

6. 

Target Acquisition 

3.8174 


1.3925 


1.0000 


6.0000 

7. 

Visual Orientation 

3.8940 


1.1835 


1.0000 


6.0000 

8. 

Responding Ground/ 

Air Radar Controller 
Instructions 

4.0047 


0.9262 


1.0000 


6.0000 

9. 

Mission Planning 

4.1046 


1.1871 


1.0000 


6.0000 

10. 

Management of TOT 

4.1187 


1.0021 


1.0000 


6.0000 

11. 

Shifting Attention to 
Targets of Opportunity 

4.1370 


1.4032 


1.0000 


6.0000 

12. 

Monitoring Flight 
Instruments 

4.3014 


0.8988 


1.0000 


6.0000 

13. 

Crew Coordination 

4.3699 


1.3864 


1.0000 


6.0000 

14. 

Type of Drop 


4.5138 


1.2152 


2.0000 



6.0000 








15. 

Munitions Deployment 

4.5160 


1.2501 


2.0000 


6.0000 

16. 

Refueling Operations 

4.5982 


0.9351 


2.0000 


6.0000 
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Combat Workload Items bv Aircraft Type; 

Table D-4 


AO/A-10 Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Threat Avoidance 

1.9118 

0.8658 

1.0000 

3.0000 

2 

In-flight Emergency 

2.0588 

0.6937 

1.0000 

4.0000 

3 

Adverse Weather 

2.2941 

0.9701 

1.0000 

5.0000 

4 

Aircraft Maneuvering 

2.3529 

1.1516 

1.0000 

5.0000 

5 

In-flight, No-Notice 
Mission Changes 

2.5882 

0.7434 

1.0000 

4.0000 



Table D-5 

AO/A-10 Combat Workload Items Found to be Moderately Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Night Operations 

2.7941 

0.8083 

1.0000 

4.0000 

2 

Night Low Level 
Navigation 

2.7941 

1.4518 

1.0000 

6.0000 

3 

Equipment Degradation 

2.8235 

0.9365 

1.0000 

5.0000 

4 

Fatigue 

2.8235 

1.1670 

1.0000 

5.0000 

5 

Target Acquisition 

2.8529 

0.8214 

1.0000 

4.0000 

6 

Threat Detection 

2.9118 

0.9001 

1.0000 

5.0000 

7 

Unfamiliar Terrain 

3.0294 

0.9996 

1.0000 

5.0000 

8 

Terrain Avoidance/ 

Terrain Following 

3.1176 

1.0945 

1.0000 

5.0000 

9 

Crew Incapacitation 

3.1176 

2.3583 

1.0000 

6.0000 

10 

Maintaining Situation 

Awareness 

3.1765 

1.1407 

1.0000 

5.0000 

11 

Low Level Navigation 

3.3235 

0.8428 

1.0000 

6.0000 

12 

Shifting Attention to 

Targets of Opportunity 

3.4706 

0.8956 

1.0000 

5.0000 

13 

Command and Control 

3.5455 

1.2271 

1.0000 

6.0000 

14 

Management of TOT 

3.5882 

0.8209 

2.0000 

5.0000 
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Table D-6 


AO/A-10 Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Managing Radio 
Communication 

3.6471 

0.8836 

2.0000 

5.0000 

2 

Responding to Ground/ 
Airborne Controller 

Instructions. 

3.7576 

0.9692 

1.0000 

5.0000 

3 

Munitions Deployment 

3.7647 

0.7808 

2.0000 

5.0000 

4 

Visual Orientation 

3.8485 

1.1214 

1.0000 

5.0000 

5 

Formation Responsibilities 

3.9706 

0.9370 

2.0000 

5.0000 

6 

Mission Plaiming 

4.0000 

1.2060 

1.0000 

6.0000 

7 

Type of Drop 

4.0588 

1.2778 

2.0000 

6.0000 

8 

Monitoring Flight 
Instruments 

4.2941 

0.9384 

1.0000 

5.0000 

9 

Refueling Operations 

4.4412 

0.7464 

3.0000 

5.0000 


Table D-7 

AO/A-10 Combat Workload Items Found to be A Little or Not Important. 

Combat Workload Item 

Mean Std. Dev. 

Minimum Maximum 

I Crew Coordination 

4.9706 1.4031 

1.0000 6.0000 
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Table D-8 


B-52 Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Crew Incapacitation 

1.7692 

1.0377 

1.0000 

5.0000 

2 

Threat Avoidance 

1.8974 

0.5523 

1.0000 

3.0000 

3 

Fatigue 

2.0513 

0.9162 

1.0000 

4.0000 

4 

Night Low Level 
Navigation 

2.3077 

1.0552 

1.0000 

6.0000 

5 

In-flight Emergency 

2.4103 

0.7152 

1.0000 

4.0000 

6 

Aircraft Maneuvering 
(Dogfight or 
avoiding the threats) 

2.4103 

1.0442 

1.0000 

6.0000 



Table D-9 

B-52 Combat Workload Items Found to be Moderately Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

In-flight, No-Notice 

Mission Changes 

2.7179 

0.6863 

2.0000 

4.0000 

2 

Adverse Weather 

2.7179 

1.1459 

1.0000 

5.0000 

3 

Terrain Avoidance/ 

Terrain Following 

2.8205 

1.0481 

1.0000 

6.0000 

4 

Night Operations 

2.8462 

1.0891 

1.0000 

6.0000 

5 

Equipment Degradation 

2.8974 

0.8206 

1.0000 

5.0000 

6 

Unfamiliar Terrain 

3.0000 

1.1239 

1.0000 

5.0000 

7 

Low Level Navigation 

3.0513 

0.8870 

2.0000 

6.0000 

8 

Maintaining Situation 

Awareness 

3.0769 

0.8701 

1.0000 

5.0000 

9 

Threat Detection 

3.2051 

0.8938 

2.0000 

5.0000 

10 

Crew Coordination 

3.3333 

1.0596 

1.0000 

5.0000 

11 

Target Acquisition 

3.4615 

0.9132 

2.0000 

5.0000 

12 

Managing Radio 
Communication 

3.4615 

0.9132 

2.0000 

5.0000 

13 

Command and Control 

3.4615 

1.0220 

1.0000 

5.0000 

14 

Formation Responsibilities 

3.5128 

0.7905 

2.0000 

5.0000 


D-5 




Table D-10 


B-52 Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Responding to Ground/ 
Airborne Controller 

Instructions. 

3.6410 

1.0127 

1.0000 

5.0000 

2 

Management of TOT 

3.6667 

0.8057 

2.0000 

5.0000 

3 

Visual Orientation 

3.6923 

1.1955 

1.0000 

6.0000 

4 

Munitions Deployment 

3.8205 

0.9423 

2.0000 

5.0000 

5 

Type of Drop 

4.0513 

0.9986 

2.0000 

6.0000 

6 

Shifting Attention to 
Targets of Opportunity 

4.0513 

1.4318 

1.0000 

6.0000 

7 

Monitoring Flight 
Instruments 

4.0769 

0.8701 

2.0000 

5.0000 

8 

Refueling Operations 

4.1282 

1.0047 

2.0000 

6.0000 

9 

Mission Planning 

4.3000 

1.0670 

2.0000 

6.0000 


Table D-11 

C-130 Combat Workload Items Found to be Distractingly Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Threat Avoidance 

2.2500 

0.8032 

1.0000 

4.0000 

2 

In-flight Emergency 

2.5000 

1.1359 

1.0000 

5.0000 

3 

Adverse Weather 

2.5625 

1.0453 

1.0000 

5.0000 

4 

Crew Incapacitation 

2.5625 

1.4797 

1.0000 

6.0000 

5 

Fatigue 

2.5938 

0.9456 

1.0000 

4.0000 
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Table D-12 


C-130 Combat Workload Items Found to be Moderately Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Aircraft Maneuvering 

2.6250 

1.0080 

1.0000 

5.0000 

2. 

In-flight, No-Notice 

Mission Changes 

2.8437 

1.1390 

1.0000 

5.0000 

3. 

Command and Control 

2.9063 

0.9625 

1.0000 

4.0000 

4. 

Terrain Avoidance/ 

Terrain Following 

2.9375 

1.0757 

1.0000 

6.0000 

5. 

Maintaining Situation 
Awareness 

2.9687 

1.1496 

1.0000 

6.0000 

6. 

Night Low Level 

Navigation 

3.0937 

1.1176 

1.0000 

6.0000 

7. 

Managing Radio 
Communication 

3.0938 

0.7771 

2.0000 

4.0000 

8. 

Threat Detection 

3.1875 

1.1760 

1.0000 

6.0000 

9. 

Equipment Degradation 

3.3125 

1.0907 

1.0000 

5.0000 

10. 

Low Level Navigation 

3.3750 

1.0395 

1.0000 

5.0000 

11. 

Unfamiliar Terrain 

3.3750 

0.9419 

1.0000 

5.0000 

12. 

Formation Responsibilities 

3.4375 

1.0140 

2.0000 

6.0000 

13. 

Night Operations 

3.4375 

0.8007 

2.0000 

5.0000 

14. 

Mission Planning 

3.5000 

1.2700 

1.0000 

6.0000 

15. 

Crew Coordination 

3.5625 

1.1341 

1.0000 

5.0000 
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C-130 Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Visual Orientation 

3.6250 

1.2889 

1.0000 

5.0000 

2 

Target Acquisition 

3.6563 

1.3586 

1.0000 

6.0000 

3 

Management of TOT 

3.8125 

0.9979 

1.0000 

5.0000 

4 

Responding to Ground/ 
Airborne Controller 

Instructions. 

3.8667 

0.9732 

2.0000 

5.0000 

5 

Monitoring Flight 

Instruments 

4.0000 

0.9837 

2.0000 

5.0000 

6 

Type of Drop 

4.0625 

1.0758 

2.0000 

6.0000 


Table D-14 


C-130 Combat Workload Items Found to be A Little or Not Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Shifting Attention to 

5.0000 

1.6461 

1.0000 

6.0000 

Targets of Opportunity 






Table D-15 

C-130 Combat Workload Items Found to be Not Applicable 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Munitions Deployment 

5.6562 

0.8654 



2 Refueling Operations 

5.7188 

0.8126 







Table D-16 


F-15 Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

I In-flight Emergency 

2.1613 

1.0984 

1.0000 


2 Crew Incapacitation 

2.2258 

1.9098 

1.0000 


3 Threat Avoidance 

2.2903 

0.7829 

1.0000 



Table D-17 

F-15 Combat Workload Items Found to be Moderately Important. 

Combat Workload Item Mean Std. Dev. Minimum 

Maximum 

1. 

Aircraft Maneuvering 

2.7419 

1.3408 

1.0000 



(Dogfight or 

2.7419 

1.3408 

1.0000 

6.0000 


avoiding the threats) 

2.7419 

1.3408 

1.0000 

6.0000 

2. 

Fatigue 

2.7742 

1.2835 

1.0000 

5.0000 

3. 

Equipment Degradation 

2.8387 

1.0032 

1.0000 

5.0000 

4. 

Adverse Weather 

2.8710 

1.2843 

1.0000 

6.0000 

5. 

In-flight, No-Notice 

2.8710 

1.0876 

1.0000 

6.0000 


Mission Changes 

2.8710 

1.0876 

1.0000 

6.0000 

6. 

Terrain Avoidance/ 

3.3226 

1.2217 

1.0000 

6.0000 


Terrain Following 

3.3226 

1.2217 

1.0000 

6.0000 

7. 

Shifting Attention to 

3.3226 

0.9447 

2.0000 

5.0000 


Targets of Opportunity 

3.3226 

0.9447 

2.0000 

5.0000 

8. 

Maintaining Situational 

3.4516 

0.9605 

2.0000 

5.0000 


Awareness 

3.4516 

0.9605 

2.0000 

5.0000 

9. 

Threat Detection 

3.4839 

0.8513 

2.0000 

5.0000 
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Table D-18 


F-15 Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Target Acquisition 

3.6452 

1.1120 

1.0000 

6.0000 

2 

Night Operations 

3.6774 

0.9447 

2.0000 

6.0000 

3 

Low Level Navigation 

3.7419 

1.1823 

2.0000 

6.0000 

4 

Night Low Level 

Navigation 

3.7419 

1.4825 

2.0000 

6.0000 

5 

Command and Control 
(such as copying and 
decoding EAMS) 

3.7419 

1.2641 

1.0000 

6.0000 

6 

Unfamiliar Terrain 

3.9355 

0.9286 

2.0000 

6.0000 

7 

Munitions Deployment 

4.0323 

0.9826 

2.0000 

5.0000 

8 

Visual Orientation 

4.0333 

1.1885 

1.0000 

6.0000 

9 

Formation Responsibilities 

4.1290 

0.9217 

3.0000 

6.0000 

10 

Mission Planning 

4.1613 

1.1575 

1.0000 

6.0000 

11 

Managing Radio 
Communication 

4.2581 

0.7732 

3.0000 

6.0000 

12 

Management of TOT 

4.2903 

0.6925 

3.0000 

5.0000 

13 

Type of Drop 

4.3333 

1.2130 

2.0000 

6.0000 

14 

Responding to Ground/ 
Airborne Controller 

Instructions. 

4.3871 

0.8032 

3.0000 

6.0000 

15 

Monitoring Flight 
Instruments 

4.4194 

0.9228 

2.0000 

6.0000 

16 

Crew Coordination 

4.4516 

1.1207 

1.0000 

6.0000 

17 

Refueling Operations 

4.5806 

0.6720 

3.0000 

6.0000 
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Table D-19 


F-16 Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 In-flight Emergency 

2.2500 

0.9675 

1.0000 


2 Threat Avoidance 

2.3636 

1.2025 

1.0000 



Table D-20 

F-16 Combat Workload Items Found to be Moderately Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Adverse Weather 

2.6364 

0.8916 

1.0000 

4.0000 

2. 

In-flight, No-Notice 

2.8409 

0.8877 

1.0000 

5.0000 


Mission Changes 

2.8409 

0.8877 

1.0000 

5.0000 

3. 

Fatigue 

2.8864 

1.1456 

1.0000 

6.0000 

4. 

Aircraft Maneuvering 

2.9091 

1.1375 

1.0000 

6.0000 


(Dogfight or 

2.9091 

1.1375 

1.0000 

6.0000 


avoiding the threats) 

2.9091 

1.1375 

1.0000 

6.0000 

5. 

Equipment Degradation 

2.9773 

0.9997 

1.0000 

5.0000 

6. 

Night Operations 

3.2727 

1.0199 

2.0000 

6.0000 

7. 

Threat Detection 

3.3409 

1.1195 

1.0000 

6.0000 

8. 

Night Low Level 

3.3636 

1.5415 

1.0000 

6.0000 


Navigation 

3.3636 

1.5415 

1.0000 

6.0000 

9. 

Target Acquisition 

3.4091 

1.1875 

2.0000 

6.0000 

10. 

Shifting Attention to 

3.4773 

0.9273 

2.0000 

5.0000 


Targets of Opportunity 

3.4773 

0.9273 

2.0000 

5.0000 

11. 

Maintaining Situation 

3.5682 

1.3364 

1.0000 

6.0000 


Awareness 

3.5682 

1.3364 

1.0000 

6.0000 

12. 

Crew Incapacitation 

3.5909 

2.2958 

1.0000 

6.0000 
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Table D-21 


F-16 Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Unfamiliar Terrain 

3.8182 

0.7857 

2.0000 

6.0000 

2 

Low Level Navigation 

3.8182 

0.9947 

2.0000 

6.0000 

3 

Terrain Avoidance/ 

Terrain Following 

3.8182 

1.2626 

2.0000 

6.0000 

4 

Visual Orientation 

4.0455 

1.2191 

1.0000 

6.0000 

5 

Munitions Deployment 

4.0682 

1.1081 

2.0000 

6.0000 

6 

Managing Radio 
Communication 

4.2045 

0.7947 

2.0000 

6.0000 

7 

Command and Control 

4.2045 

1.1729 

2.0000 

6.0000 

8 

Responding to Ground/ 
Airborne Controller 

Instructions. 

4.2143 

0.8421 

2.0000 

6.0000 

9 

Management of TOT 

4.2500 

0.8105 

2.0000 

6.0000 

10 

Mission Planning 

4.2500 

1.3316 

1.0000 

6.0000 

11 

Formation Responsibilities 

4.2955 

0.9296 

2.0000 

6.0000 

12 

Monitoring Flight 
Instruments 

4.5227 

0.8757 

2.0000 

6.0000 

13 

Type of Drop 

4.5909 

1.0414 

2.0000 

6.0000 




Table D-22 




F-16 Combat Workload Items Found to be A Little 

or Not Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Refueling Operations 

4.6136 

0.6893 

3.0000 

6.0000 




Table D-23 




F-16 Combat Workload Items Found to be Not Applicable 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Crew Coordination 

5.7273 

0.7270 

3.0000 

6.0000 
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Table D-24 


KC-135 Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Crew Incapacitation 

2.3590 

1.5129 

1.0000 

6.0000 

2 

In-flight Emergency 

2.4359 

0.9402 

1.0000 

5.0000 

3 

Threat Avoidance 

2.5385 

1.0475 

1.0000 

5.0000 


Table D-25 

KC-135 Combat Workload Items Found to be Moderately Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Fatigue 

2.6667 

1.0596 

1.0000 

5.0000 

2. 

Adverse Weather 

2.8462 

0.8747 

1.0000 

5.0000 

3. 

Equipment Degradation 

3.0000 

1.0513 

1.0000 

6.0000 

4. 

In-flight, No-Notice 

3.2051 

0.8639 

2.0000 

5.0000 


Mission Changes 

3.2051 

0.8639 

2.0000 

5.0000 

5. 

Formation Responsibilities 

3.3333 

0.8686 

2.0000 

5.0000 

6. 

Aircraft Maneuvering 

3.5128 

1.6523 

1.0000 

6.0000 


(Dogfight or 

3.5128 

1.6523 

1.0000 

6.0000 


avoiding the threats) 

3.5128 

1.6523 

1.0000 

6.0000 
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Table D-26 


KC-135 Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Command and Control 

3.6154 

0.9351 

2.0000 

5.0000 

2 

Maintaining Situation 
Awareness 

3.6410 

1.1582 

1.0000 

6.0000 

3 

Managing Radio 
Communication 

3.7949 

1.0558 

2.0000 

5.0000 

4 

Refueling Operations 

3.7949 

1.0558 

2.0000 

5.0000 

5 

Threat Detection 

3.8462 

1.7702 

1.0000 

6.0000 

6 

Crew Coordination 

3.9487 

1.1227 

1.0000 

5.0000 

7 

Night Operations 

4.0769 

0.8701 

2.0000 

6.0000 

8 

Visual Orientation 

4.0769 

1.0854 

1.0000 

6.0000 

9 

Responding to Ground/ 
Airborne Controller 

Instructions. 

4.1622 

0.7643 

3.0000 

5.0000 

10 

Mission Planning 

4.2821 

0.9445 

2.0000 

6.0000 

11 

Unfamiliar Terrain 

4.4103 

1.0691 

2.0000 

6.0000 

12 

Monitoring Flight 
Instruments 

4.4359 

0.7538 

2.0000 

5.0000 
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Table D-27 


KC-135 Combat Workload Items Found to be A Little or Not Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Terrain Avoidance/ 

Terrain Following 

4.5641 

1.6025 

1.0000 

6.0000 

2 

Night Low Level 
Navigation 

4.7949 

1.9219 

1.0000 

6.0000 

3 

Management of TOT 

5.0000 

1.1002 

2.0000 

6.0000 

4 

Low Level Navigation 

5.0000 

1.6222 

1.0000 

6.0000 

5 

Shifting Attention to 
Targets of Opportunity 

5.4872 

1.0729 

2.0000 

6.0000 




Table D-28 




KC-135 Combat Workload Items Found to be Not Applicable 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Target Acquisition 

5.7436 

0.7853 

3.0000 


2 

Type of Drop 

5.7949 

0.6561 

3.0000 

6.0000 

3 

Munitions Deployment 

5.8205 

0.7208 


6.0000 
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Combat Workload Items hv Mission Type: 

Table D-29 


Conv. Hvy. Bomb. Missions' Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Threat Avoidance 

1.8611 

0.5929 

1.0000 

3.0000 

2 

Crew Incapacitation 

1.8611 

1.2684 

1.0000 

6.0000 

3 

Fatigue 

2.0556 

0.9545 

1.0000 

4.0000 

4 

Night Low Level 
Navigation 

2.2778 

1.0853 

1.0000 

6.0000 

5 

In-flight Emergency 

2.3611 

0.6825 

1.0000 

4.0000 

6 

Aircraft Maneuvering 
(Dogfight or 
avoiding the threats) 

2.4444 

1.0541 

1.0000 

6.0000 
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Table D-30 


Con. Hvy. Bomb. Missions’ Combat Workload Items Found to be Moderately Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Adverse Weather 

2.6667 

1.2189 

1.0000 

5.0000 

2. 

In-flight, No-Notice 

Mission Changes 

2.6944 

0.7099 

2.0000 

4.0000 

3. 

Terrain Avoidance/ 

Terrain Following 

2.8056 

1.0907 

1.0000 

6.0000 

4. 

Equipment Degradation 

2.8611 

0.8333 

1.0000 

5.0000 

5. 

Night Operations 

2.8611 

1.1251 

1.0000 

6.0000 

6. 

Low Level Navigation 

2.9722 

0.9098 

2.0000 

6.0000 

7. 

Maintaining Situation 

Awareness 

3.0000 

0.8619 

1.0000 

5.0000 

8. 

Unfamiliar Terrain 

3.0278 

1.0552 

1.0000 

5.0000 

9. 

Threat Detection 

3.2222 

0.9292 

2.0000 

5.0000 

10. 

Crew Coordination 

3.3056 

1.0907 

1.0000 

5.0000 

11. 

Target Acquisition 

3.3889 

0.9644 

2.0000 

5.0000 

12. 

Command and Control 

3.3889 

1.0764 

1.0000 

5.0000 

13. 

Managing Radio 
Communication 

3.4167 

0.9063 

2.0000 

5.0000 

14. 

Formation Responsibilities 

3.4722 

0.8102 

2.0000 

5.0000 

15. 

Responding to Ground/ 
Airborne Controller 

Instructions. 

3.5833 

1.0247 

1.0000 

5.0000 
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Table D-31 


Con. Hvy. Bomb. Missions' Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

I. 

Management of TOT 

3.6111 

0.8028 

2.0000 

5.0000 

2 

Visual Orientation 

3.6389 

1.2225 

1.0000 

6.0000 

3 

Munitions Deployment 

3.8333 

0.9710 

2.0000 

5.0000 

4 

Type of Drop 

3.9167 

0.9673 

2.0000 

6.0000 

5 

Shifting Attention to 
Targets of Opportunity 

3.9167 

1.4417 

1.0000 

6.0000 

6 

Monitoring Flight 
Instruments 

4.0278 

0.9098 

2.0000 

5.0000 

7 

Refueling Operations 

4.1667 

0.9710 

2.0000 

6.0000 

8 

Mission Planning 

4.2432 

1.0905 

2.0000 

6.0000 
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Table D-32 


Theater Airlift Missions' Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Threat Avoidance 





2 

In-flight Emergency 





3 

Crew Incapacitation 

2.3077 



5.0000 

4 

Adverse Weather 

2.3846 

0.7679 


4.0000 



Table D-33 

Theater Airlift Missions' Combat Workload Items Found to be Moderately Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Fatigue 

2.6154 

0.9608 

1.0000 

4.0000 

2. 

Aircraft Maneuvering 

2.6923 

0.9473 

1.0000 

4.0000 

3. 

In-flight, No-Notice 

Mission Changes 

2.6923 

1.0316 

1.0000 

5.0000 

4. 

Command and Control 

2.8462 

0.8987 

2.0000 

4.0000 

5. 

Managing Radio 
Communication 

3.0000 

0.7071 

2.0000 

4.0000 

6. 

Maintaining Situation 

Awareness 

3.0000 

0.8165 

1.0000 

4.0000 

7. 

Equipment Degradation 

3.0769 

1.0377 

1.0000 

4.0000 

8. 

Night Low Level 

Navigation 

3.0769 

1.1152 

2.0000 

6.0000 

9. 

Threat Detection 

3.1538 

0.8006 

2.0000 

4.0000 

10. 

Terrain Avoidance/ 

Terrain Following 

3.1538 

1.0682 

2.0000 

6.0000 

11. 

Low Level Navigation 

3.2308 

0.8321 

2.0000 

5.0000 

12. 

Unfamiliar Terrain 

3.4615 

0.6602 

2.0000 

4.0000 

13. 

Formation Responsibilities 

3.5385 

0.9674 

2.0000 

6.0000 

14. 

Visual Orientation 

3.5385 

1.3301 

1.0000 

5.0000 
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Table D-34 


Theater Airlift Missions' Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item Mean Std. Dev. Minimum Maximum 


1 

Night Operations 

3.6154 

0.7679 

3.0000 

5.0000 

2 

Crew Coordination 

3.6154 

0.9608 

2.0000 

5.0000 

3 

Target Acquisition 

3.6154 

1.2609 

2.0000 

6.0000 

4 

Mission Planning 

3.6923 

1.3775 

2.0000 

6.0000 

5 

Management of TOT 

3.7692 

0.5991 

3.0000 

5.0000 

6 

Monitoring Flight 
Instruments 

3.9231 

0.8623 

3.0000 

5.0000 

7 

Responding to Ground/ 
Airborne Controller 

Instructions. 

4.0000 

0.7746 

3.0000 

5.0000 

8 

Type of Drop 

4.2308 

0.9268 

3.0000 

6.0000 


Table D-35 


Theater Airlift Missions' Combat Workload Items Found to be A Little or Not Important 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Shifting Attention to 

5.0000 

1.6330 

2.0000 

6.0000 

Targets of Opportunity 






Table D-36 


Theater Airlift Missions' Combat Workload Items Found to be Not Applicable 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Munitions Deployment 

5.6923 

0.6304 



2 Refueling Operations 

5.6923 

0.8549 
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Table D-37 


Tactical Airlift Missions’ Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Threat Avoidance 

2.2778 

0.8264 

1.0000 

3.0000 

2 Fatigue 

2.4444 

0.9218 

1.0000 

4.0000 

3 In-flight Emergency 

2.5556 

1.1991 

1.0000 

5.0000 



Table D-38 

Tactical Airlift Missions' Combat Workload Items Found to be Moderately Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Adverse Weather 

2.6667 

1.1376 

1.0000 

5.0000 

2. 

Aircraft Maneuvering 

2.6667 

1.1376 

1.0000 

5.0000 

3. 

Crew Incapacitation 

2.6667 

1.7150 

1.0000 

6.0000 

4. 

Terrain Avoidance/ 

Terrain Following 

2.7778 

1.1144 

1.0000 

4.0000 

5. 

Maintaining Situation 

Awareness 

2.7778 

1.2154 

1.0000 

6.0000 

6. 

Command and Control 

2.8333 

1.0432 

1.0000 

4.0000 

7. 

Night Low Level 

Navigation 

3.0000 

1.0847 

1.0000 

5.0000 

8. 

In-flight, No-Notice 

Mission Changes 

3.0000 

1.2367 

1.0000 

5.0000 

9. 

Threat Detection 

3.0556 

1.2590 

1.0000 

6.0000 

10. 

Managing Radio 

Communication 

3.1111 

0.8324 

2.0000 

4.0000 

11. 

Mission Planning 

3.2222 

1.1660 

1.0000 

5.0000 

12. 

Night Operations 

3.2778 

0.7519 

2.0000 

4.0000 

13. 

Unfamiliar Terrain 

3.3333 

1.1376 

1.0000 

5.0000 

14. 

Equipment Degradation 

3.3889 

1.0922 

1.0000 

5.0000 

15. 

Low Level Navigation 

3.3889 

1.1448 

1.0000 

5.0000 

16. 

Formation Responsibilities 

3.4444 

1.0416 

2.0000 

5.0000 

17. 

Crew Coordination 

3.4444 

1.2472 

1.0000 

5.0000 

18. 

Visual Orientation 

3.5556 

1.2935 

1.0000 

5.0000 
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Table D-39 


Tactical Airlift Missions’ Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Target Acquisition 

3.7222 

1.4473 

1.0000 

6.0000 

2 

Responding to Ground/ 
Airborne Controller 

Instructions. 

3.7778 

1.0603 

2.0000 

5.0000 

3 

Management of TOT 

3.8889 

1.2314 

1.0000 

5.0000 

4 

Type of Drop 

3.9444 

1.2113 

2.0000 

6.0000 

5 

Monitoring Flight 
Instruments 

4.0000 

1.0847 

2.0000 

5.0000 




Table D-40 




Tactical Airlift Missions' Combat Workload Items Found to be A Little or Not Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Shifting Attention to 

Targets of Opportunity 

4.9444 

1.7311 

1.0000 

6.0000 




Table D-41 




Tactical Airlift Missions' Combat Workload Items Found to be Not Applicable. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Munitions Deployment 

5.6111 

1.0369 


6.0000 

2 

Refueling Operations 

5.8889 

0.4714 


6.0000 
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Table D-42 


A/A Refueling Missions' Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Crew Incapacitation 

2.4324 

1.5191 



2 In-flight Emergency 

2.4865 

0.9316 



3 Threat Avoidance 

2.5676 

1.0682 


5.0000 



Table D-43 

A/A Refueling Missions' Combat Workload Items Found to be Moderately Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Fatigue 

2.6757 

1.0815 

1.0000 

5.0000 

2. 

Adverse Weather 

2.8649 

0.8870 

1.0000 

5.0000 

3. 

Equipment Degradation 

3.0000 

1.0801 

1.0000 

6.0000 

4. 

In-flight, No-Notice 

Mission Changes 

3.2432 

0.8630 

2.0000 

5.0000 

5. 

Formation Responsibilities 

3.3514 

0.8569 

2.0000 

5.0000 

6. 

Aircraft Maneuvering 
(Dogfight or 
avoiding the threats) 

3.5405 

1.6765 

1.0000 

6.0000 
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Table D-44 


A/A Refueling Missions' Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Maintaining Situation 
Awareness 

3.6216 

1.1868 

1.0000 

6.0000 

2 

Command and Control 

3.6757 

0.9145 

2.0000 

5.0000 

3 

Managing Radio 
Communication 

3.8378 

1.0675 

2.0000 

5.0000 

4 

Threat Detection 

3.8378 

1.7562 

1.0000 

6.0000 

5 

Crew Coordination 

3.9459 

1.1291 

1.0000 

5.0000 

6 

Night Operations 

4.0811 

0.8938 

2.0000 

6.0000 

7 

Visual Orientation 

4.1351 

1.0843 

1.0000 

6.0000 

8 

Responding to Ground/ 
Airborne Controller 

Instructions. 

4.1714 

0.7854 

3.0000 

5.0000 

9 

Refueling Operations 

4.2973 

0.8119 

3.0000 

5.0000 

10 

Mission Planning 

4.3514 

0.9194 

2.0000 

6.0000 

11 

Unfamiliar Terrain 

4.4054 

1.0919 

2.0000 

6.0000 

12 

Monitoring Flight 
Instruments 

4.4595 

0.7672 

2.0000 

5.0000 



Table D-45 

A/A Refueling Missions' Combat Workload Items Found to be A Little or Not Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Terrain Avoidance/ 

Terrain Following 

4.6486 

1.5849 

1.0000 

6.0000 

2 

Night Low Level 
Navigation 

4.8108 

1.9413 

1.0000 

6.0000 

3 

Management of TOT 

5.0000 

1.1055 

2.0000 

6.0000 

4 

Low Level Navigation 

5.0270 

1.6242 

1.0000 

6.0000 

5 

Shifting Attention to 
Targets of Opportunity 

5.4595 

1.0953 

2.0000 

6.0000 
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Table D-46 


A/A Refueling Missions' Combat Workload Items Found to be Not Applicable. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Target Acquisition 

5.8108 

0.6599 

3.0000 

6.0000 

2 Type of Drop 

5.8108 

0.6599 

3.0000 

6.0000 

3 Munitions Deployment 

5.8108 

0.7393 

2.0000 

6.0000 
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Table D-47 


Air Superiority Missions' Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Fatigue 


1.0690 

1.0000 

4.0000 

2 Threat Avoidance 


0.7071 

1.0000 

3.0000 

3 In-flight Emergency 


1.1877 

1.0000 

5.0000 

4 Adverse Weather 


1.3093 

1.0000 

4.0000 



Table D-48 

Air Superiority Missions' Combat Workload Items Found to be Moderately Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Terrain Avoidance/ 

Terrain Following 

2.7500 

1.2817 

1.0000 

5.0000 

2. 

Crew Incapacitation 

2.7500 

2.4349 

1.0000 

6.0000 

3. 

Night Operations 

3.0000 

0.7559 

2.0000 

4.0000 

4. 

In-flight, No-Notice 
Mission Changes 

3.1250 

0.9910 

2.0000 

5.0000 

5. 

Aircraft Maneuvering 
(Dogfight or 
avoiding the threats) 

3.1250 

1.2464 

2.0000 

5.0000 

6. 

Threat Detection 

3.3750 

1.0607 

2.0000 

5.0000 

7. 

Low Level Navigation 

3.5000 

1.3093 

2.0000 

6.0000 
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Table D-49 



Air Superiority Missions’ Combat Workload Items Found to be Somewhat Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Equipment Degradation 

3.6250 

0.7440 

3.0000 

5.0000 

2 

Shifting Attention to 

Targets of Opportunity 

3.7500 

0.8864 

3.0000 

5.0000 

3 

Maintaining Situation 

Awareness 

3.8750 

0.9910 

3.0000 

5.0000 

4 

Command and Control 
(such as copying and 
decoding EAMS) 

3.8750 

1.3562 

2.0000 

6.0000 

5 

Unfamiliar Terrain 

4.0000 

1.0690 

2.0000 

5.0000 

6 

Managing Radio 
Communication 

4.1250 

0.6409 

3.0000 

5.0000 

7 

Munitions Deployment 

4.2500 

0.8864 

3.0000 

5.0000 

8 

Formation Responsibilities 

4.2500 

1.0351 

3.0000 

5.0000 

9 

Mission Planning 

4.2500 

1.0351 

3.0000 

5.0000 

10 

Target Acquisition 

4.2500 

1.0351 

3.0000 

6.0000 


Monitoring Flight 

Instruments 

4.3750 

1.0607 

2.0000 

5.0000 

11 

Night Low Level 

Navigation 

4.5000 

1.8516 

2.0000 

6.0000 

12 

Visual Orientation 

4.5714 

0.7868 

3.0000 

5.0000 


Table D-50 

Air Interdiction Missions' Combat Workload Items Found to be A Little or No Important. 


Combat Workload Item 

Mean 

Std. E>ev. 

Minimum 

Maximum 

Management of TOT 

4.7500 

0.4629 

4.0000 

5.0000 


- 

Responding to Ground/ 
Airborne Controller 

Instructions. 

4.7500 

0.4629 

4.0000 

5.0000 


Refueling Operations 

4.8750 

0.3536 

4.0000 

5.0000 


Type of Drop 

5.5714 

0.7868 

4.0000 

6.0000 


D-27 








Table D-51 


Air Interdiction Missions’ Combat Workload Items Found to be Not Applicable. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Crew Coordination 

5.6250 

0.7440 

4.0000 

6.0000 
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Table D-52 


CAS Missions' Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Threat Avoidance 

1.8462 

0.8339 

1.0000 

3.0000 

2 In-flight Emergency 

2.1154 

0.7114 

1.0000 

4.0000 

3 Adverse Weather 

2.3462 

1.0561 

1.0000 

5.0000 

4 Aircraft Maneuvering 
(Dogfight or 
avoiding the threats) 

2.3846 

1.0612 

1.0000 

4.0000 

5 In-flight, No-Notice 
Mission Changes 

2.5385 

0.8115 

1.0000 

4.0000 



Table D-53 

CAS Missions’ Combat Workload Items Found to be Moderately Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Fatigue 

2.8077 

1.0961 

1.0000 

5.0000 

2. 

Night Low Level 
Navigation 

2.8077 

1.4702 

1.0000 

6.0000 

3. 

Night Operations 

2.8462 

0.7317 

2.0000 

4.0000 

4. 

Equipment Degradation 

2.8462 

1.0077 

1.0000 

5.0000 

5. 

Threat Detection 

2.9231 

1.0168 

1.0000 

6.0000 

6. 

Target Acquisition 

2.9615 

0.8709 

2.0000 

5.0000 

7. 

Terrain Avoidance/ 

Terrain Following 

3.0385 

1.1129 

1.0000 

5.0000 

8. 

Crew Incapacitation 

3.0385 

2.2712 

1.0000 

6.0000 

9. 

Unfamiliar Terrain 

3.0769 

0.9348 

1.0000 

5.0000 

10. 

Maintaining Situation 

Awareness 

3.1923 

1.2967 

1.0000 

6.0000 

11. 

Low Level Navigation 

3.3462 

0.8918 

1.0000 

6.0000 

12. 

Shifting Attention to 

Targets of Opportunity 

3.4615 

0.8593 

1.0000 

5.0000 

13. 

Management of TOT 

3.5000 

0.8124 

2.0000 

5.0000 
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Table D-54 


CAS Missions' Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Command and Control 
(such as copying and 
decoding EAMS) 

3.6000 

1.2583 

1.0000 

6.0000 

2 

Munitions Deployment 

3.7692 

0.9081 

2.0000 

6.0000 

3 

Managing Radio 
Communication 

3.7692 

0.9923 

2.0000 

5.0000 

4 

Responding to Ground/ 
Airborne Controller 

Instructions. 

3.8000 

0.9574 

1.0000 

5.0000 

5 

Visual Orientation 

3.8800 

1.2356 

1.0000 

6.0000 

6 

Formation Responsibilities 

3.9615 

0.9992 

2.0000 

5.0000 

7 

Mission Planning 

4.0000 

1.1662 

1.0000 

6.0000 

8 

Type of Drop 

4.1923 

1.2967 

2.0000 

6.0000 

9 

Monitoring Flight 
Instruments 

4.4231 

0.8566 

3.0000 

6.0000 

10 

Refueling Operations 

4.5385 

0.6469 

3.0000 

5.0000 


Table D-55 

CAS Missions' Combat Workload Items Found to be A Little or No Important. 

Combat Workload Item 

Mean Std. Dev. 

Minimum 

Maximum 

1 Crew Coordination 

5.1154 1.3950 

1.0000 

6.0000 
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Table D-56 


Air Interdiction Missions' Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

In-flight Emergency 

2.2593 

1.0494 

1.0000 


2 

Threat Avoidance 

2.4259 

1.0569 

1.0000 





Table D-57 




Air Interdiction Missions' Combat Workload Items Found to be Moderately Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

Aircraft Maneuvering 
(Dogfight or 
avoiding the threats) 

2.7407 

1.3202 

1.0000 

6.0000 

2. 

Adverse Weather 

2.8148 

1.0108 

1.0000 

6.0000 

3. 

Equipment Degradation 

2.8519 

0.9398 

1.0000 

5.0000 

4. 

In-flight, No-Notice 
Mission Changes 

2.9259 

0.9286 

1.0000 

6.0000 

5. 

Crew Incapacitation 

2.9444 

2.1841 

1.0000 

6.0000 

6. 

Fatigue 

2.9815 

1.2052 

1.0000 

6.0000 

7. 

Night Low Level 
Navigation 

3.3333 

1.2439 

2.0000 

6.0000 

8. 

Target Acquisition 

3.3519 

1.1186 

1.0000 

6.0000 

9. 

Shifting Attention to 

Targets of Opportunity 

3.4259 

0.9235 

2.0000 

5.0000 

10. 

Threat Detection 

3.4259 

0.9437 

2.0000 

6.0000 

11. 

Maintaining Situation 

Awareness 

3.5370 

1.1770 

1.0000 

6.0000 

12. 

Night Operations 

3.5556 

1.0031 

2.0000 

6.0000 
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Table D-58 


Air Interdiction Missions' Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 

Terrain Avoidance/ 

Terrain Following 

3.6296 

1.0692 

2.0000 

6.0000 

2 

Unfamiliar Terrain 

3.7778 

0.9842 

1.0000 

6.0000 

3 

Low Level Navigation 

3.7963 

0.9393 

2.0000 

6.0000 

4 

Munitions Deployment 

4.0000 

1.0279 

2.0000 

6.0000 

5 

Command and Control 
(such as copying and 
decoding HAMS) 

4.0000 

1.2439 

1.0000 

6.0000 

6 

Visual Orientation 

4.0556 

1.1396 

1.0000 

6.0000 

7 

Managing Radio 
Communication 

4.1667 

0.7709 

3.0000 

6.0000 

8 

Management of TOT 

4.1852 

0.7542 

2.0000 

6.0000 

9 

Responding to Ground/ 
Airborne Controller 

Instructions. 

4.1923 

0.8641 

2.0000 

6.0000 

10 

Formation Responsibilities 

4.2222 

0.8831 

2.0000 

6.0000 

11 

Mission Planning 

4.2407 

1.3022 

1.0000 

6.0000 

12 

Type of Drop 

4.2963 

1.0751 

2.0000 

6.0000 

13 

Monitoring Flight 
Instruments 

4.4630 

0.8625 

2.0000 

6.0000 

14 

Refueling Operations 

4.5741 

0.7423 

3.0000 

6.0000 


Table D-59 

Air Interdiction Missions’ Combat Workload Items Found to be A Little or Not Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 Crew Coordination 

4.8333 

1.1935 

1.0000 

6.0000 
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Table D-60 


CAP Missions' Combat Workload Items Found to be Distractingly Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1 In-flight Emergency 


0.9428 

1.0000 

4.0000 

2 Threat Avoidance 


1.0593 

1.0000 

4.0000 



Table D-61 

CAP Missions' Combat Workload Items Found to be Moderately Important. 

Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 

1. 

In-flight, No-Notice 
Mission Changes 

2.7000 

0.6749 

2.0000 

4.0000 

2. 

Adverse Weather 

2.8000 

1.0328 

2.0000 

5.0000 

3. 

Equipment Degradation 

2.8000 

1.3166 

1.0000 

5.0000 

4. 

Fatigue 

3.0000 

1.3333 

1.0000 

5.0000 

5. 

Threat Detection 

3.2000 

1.2293 

1.0000 

5.0000 

6. 

Aircraft Maneuvering 
(Dogfight or 
avoiding the threats) 

3.2000 

1.2293 

1.0000 

5.0000 

7. 

Shifting Attention to 

Targets of Opportunity 

3.3000 

0.6749 

2.0000 

4.0000 

8. 

Crew Incapacitation 

3.4000 

2.5473 

1.0000 

6.0000 

9. 

Unfamiliar Terrain 

3.5000 

0.8498 

2.0000 

5.0000 
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Table D-62 



CAP Missions' Combat Workload Items Found to be Somewhat Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 


1 

Command and Control 

3.6000 

1.0750 

2.0000 

6.0000 



(such as copying and 
decoding EAMS) 






2 

Night Operations 

3.6000 

1.0750 

2.0000 

6.0000 


3 

Visual Orientation 

3.6000 

1.1738 

1.0000 

5.0000 


4 

Maintaining Situation 
Awareness 

3.7000 

0.9487 

2.0000 

5.0000 


5 

Target Acquisition 

3.7000 

1.1595 

2.0000 

5.0000 


6 

Night Low Level 

Navigation 

3.8000 

1.8738 

1.0000 

6.0000 


7 

Low Level Navigation 

4.1000 

1.2867 

2.0000 

6.0000 


8 

Mission Planning 

4.3000 

0.8233 

3.0000 

5.0000 


9 

Managing Radio 
Communication 

4.3000 

0.9487 

2.0000 

5.0000 


10 

Refueling Operations 

4.3000 

0.9487 

2.0000 

5.0000 


11 

Munitions Deployment 

4.3000 

1.1595 

2.0000 

6.0000 


12 

Type of Drop 

4.3000 

1.2517 

2.0000 

6.0000 


13 

Formation Responsibilities 

4.4000 

0.8433 

3.0000 

5.0000 


14 

Responding to Ground/ 
Airborne Controller 

Instructions. 

4.5000 

0.7071 

4.0000 

6.0000 


15 

Terrain Avoidance/ 

4.5000 

1.3540 

2.0000 

6.0000 



Terrain Following 

4.5000 

1.3540 

2.0000 

6.0000 





Table D-63 





CAP Missions’ Combat Workload Items Found to be A Little or Not Important. 


Combat Workload Item 

Mean 

Std. Dev. 

Minimum 

Maximum 


1 

Management of TOT 

4.6000 

0.6992 

3.0000 

5.0000 

m 

2 

Monitoring Flight 

Instruments 

4.7000 

0.4830 

4.0000 

5.0000 

> 
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Table D-64 


CAP Missions' Combat Workload Items Found to be Not Applicable 


Combat Workload Item Mean Std. Dev. Minimum Maximum 


a 

1 Crew Coordination 5.7000 0.4830 5.0000 6.0000 
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Miscellaneous Data; 


Table D-65a 


Initial Factor Method: Principal Components 

Factor Pattern 


Item # 

Combat Workload Items 

Factor 1 

Factor 2 

Q16 

Night Low Level Navigation 

0.77231 

0.3244 

Q15 

Low Level Navigation 

0.74182 

0.15369 

Q29 

Crew Coordination 

0.73364 

0.08411 

Q25 

In-flight Emergency 

0.67657 

-0.30689 

Q33 

Night Operations 

0.66906 

0.36405 

Q34 

Unfamiliar Terrain 

0.66027 

0.20675 

QIO 

Terrain Avoidance/Terrain Following 

0.65754 

0.19341 

Q18 

Formation Responsibilities 

0.65611 

0.12035 

Q37 

Responding to Ground/Airbome 

0.59703 

0.01568 


Controller Instructions. 

0.59703 

0.01568 

Q20 

In-flight, No-Notice Mission Changes 

0.59356 

-0.32371 

Q19 

Management of TOT 

0.59326 

-0.03848 

Q30 

Aircraft Maneuvering 

0.57372 

-0.04129 

Q26 

Visual Orientation 

0.56825 

-0.04454 

Q14 

Equipment Degradation 

0.55536 

-0.18587 

Q31 

Target Acquisition 

0.55124 

0.03632 

Q17 

Threat Avoidance 

0.53988 

0.13651 

Q32 

Type of Drop 

0.53782 

-0.19364 

Q22 

Munitions Deployment 

0.53643 

-0.27001 



Initial Factor Method: Principal Components 


Factor Pattern 


Item # 

Combat Workload Items 

Factor 1 

Factor 2 

Qll 

Maintaining Situation Awareness 

0.5331 

-0.1363 

Q24 

Crew Incapacitation 

0.5056 

-0.2816 

Q36 

Refueling Operations 

0.5049 

-0.0790 

Q28 

Fatigue 

0.4894 

-0.0199 

Q23 

Threat Detection 

0.4523 

-0.0227 

Q27 

Command and Control 

0.4142 

-0.3795 

Q35 

Managing Radio Communication 

0.4008 

0.0404 

Q21 

Shifting Attention to 

0.3845 

-0.6094 


Targets of Opportunity 



Q12 

Adverse Weather 

0.3635 

0.0828 

Q9 

Mission Planning 

0.2832 

-0.0513 

Q13 

Monitoring Flight Instruments 

0.2831 

0.0552 

Q42 

Combat versus Peace time 

0.2590 

0.0544 

Q41 

Updating the Regulations 

0.2299 

0.5101 

Q43 

Simulator Effectiveness 

0.1679 

0.3826 

Q47 

Volunteering Sim Exp.s 

0.1447 

0.3603 

Q39 

Modifying the Cockpit 

0.0838 

0.3985 

Q46 

Volunteering Combat Exp.s 

0.0346 

0.4744 

Q44 

US and Allies’ Air Power 

0.0340 

0.5416 

Q40 

Technological Innovations 

0.0255 

0.5674 

Q45 

Combat Realities 

-0.0121 

0.2766 

Q38 

Additional Crew Member 

-0.0206 

-0.0097 



Factor 1 

Factor 2 


Variance Explained by Each Factor 

9.273758 

2.982069 





Reliability Test of CWL 


Correlation Analysis —Cronbach Coefficient Alpha 
for RAW variables : 0.920502 

for STANDARDIZED variables: 0.922903 


Raw Variables Std. Variables 


Item # 

Deleted Variables 

Combat Workload Factors 

Correlation 

with Total 

Alpha 

Correlation 

with Total 

Alpha 

Q15 

Low Level Navigation 

0.6263 

0.9163 

0.6300 

0.9186 

Q29 

Crew Coordination 

0.6188 

0.9161 

0.6173 

0.9188 

Q19 

Management of TOT 

0.6186 

0.9165 

0.6196 

0.9188 

Q33 

Night Operations 

0.6108 

0.9164 

0.6165 

0.9189 

Q32 

Type of Drop 

0.6065 

0.9165 

0.6160 

0.9189 

Q16 

Night Low Level Navigation 

0.5972 

0.9165 

0.6025 

0.9191 

Q31 

Target Acquisition 

0.5859 

0.9167 

0.5973 

0.9192 

Q37 

Responding to Ground/ Airborne 
Controller Instructions 

0.5775 

0.9170 

0.5757 

0.9195 

Q18 

Formation Responsibilities 

0.5749 

0.9169 

0.5788 

0.9194 

Q20 

In-flight, No-Notice 

Mission Changes 

0.5681 

0.9171 

0.5711 

0.9196 

Q17 

Threat Avoidance 

0.5678 

0.9171 

0.5764 

0.9195 

Q30 

Aircraft Maneuvering 

0.5638 

0.9171 

0.5696 

0.9196 

Q26 

Visual Orientation 

0.5596 

0.9172 

0.5558 

0.9198 

Q34 

Unfamiliar Terrain 

0.5547 

0.9172 

0.5576 

0.9198 

QIO 

Terrain Avoidance/ Following 

0.5418 

0.9174 

0.5442 

0.9200 

Q28 

Fatigue 

0.5279 

0.9177 

0.5265 

0.9203 

Q25 

In-flight Emergency 

0.5205 

0.9178 

0.5164 

0.9204 

Q14 

Equipment Degradation 

0.4943 

0.9182 

0.4912 

0.9208 

Qll 

Maintaining Situation Awareness 

0.4791 

0.9185 

0.4852 

0.9209 

Q21 

Shifting Attention to 

Targets of Opportunity 

0.4659 

0.9186 

0.4749 

0.9210 

Q27 

Command and Control 

0.4614 

0.9187 

0.4586 

0.9213 

Q23 

Threat Detection 

0.4607 

0.9187 

0.4687 

0.9211 

Q35 

Managing Radio 

Communication 

0.4588 

0.9187 

0.4640 

0.9212 

Q22 

Munitions Deployment 

0.4547 

0.9188 

0.4664 

0.9212 

Q12 

Adverse Weather 

0.4030 

0.9197 

0.3996 

0.9222 

Q36 

Refueling Operations 

0.4001 

0.9194 

0.4049 

0.9221 

Q24 

Crew Incapacitation 

0.3767 

0.9209 

0.3790 

0.9225 



Table D-67 


Result of Bonferroni Procedure-Comparison of Aircraft's Mean CWL Scores 


Aircraft 

Comparison 

Lower 

Confidence 

Limit 

Difference 

Between 

Means 

Upper 

Confidence 

Limit 

Significance 

F-15 

-F-16 

-8.846 

1.985 

12.816 


F-15 

-B-52 

-4.945 

5.675 

16.294 


F-15 

-A-10 

-4.080 

6.911 

17.902 


F-15 

- C-130 

1.058 

12.228 

23.398 


F-15 

- KC-135 

6.922 

17.477 

28.033 

*** 

F-16 

-F-15 

-12.816 

-1.985 

8.846 


F-16 

.B-52 

-6.671 

3.690 

14.051 


F-16 

-A-10 

-5.816 

4.926 

15.668 


F-16 

- C-130 

-0.682 

10.243 

21.168 


F-16 

- KC-135 

5.196 

15.492 

25.788 

*** 

B-52 

-F-15 

-16.294 

-5.675 

4.945 


B-52 

- F-16 

-14.051 

-3.690 

6.671 


B-52 

-A-10 

-9.292 

1.236 

11.765 


B-52 

- C-130 

-4.162 

6.553 

17.268 


B-52 

- KC-135 

1.729 

11.802 

21.875 

*** 

A-10 

-F-15 

-17.902 

-6.911 

4.080 


A-IO 

-F-16 

-15.668 

-4.926 

5.816 


A-10 

-B-52 

-11.765 

-1.236 

9.292 


A-10 

- C-130 

-5.767 

5.317 

16.400 


A-10 

- KC-135 

0.102 

10.566 

21.030 

*** 

C-130 

.F-15 

-23.398 

-12.228 

-1.058 

*** 

C-130 

-F-16 

-21.168 

-10.243 

0.682 


C-130 

-B-52 

-17.268 

-6.553 

4.162 


C-130 

- A-10 

-16.400 

-5.317 

5.767 


C-130 

- KC-135 

-5.403 

5.249 

15.901 


KC-135 

-F-15 

-28.033 

-17.477 

-6.922 

*** 

KC-135 

-F-16 

-25.788 

-15.492 

-5.196 


KC-135 

-B-52 

-21.875 

-11.802 

-1.729 


KC-135 

-A-10 

-21.030 

-10.566 

-0.102 

*** 

KC-135 

- C-130 

-15.901 

-5.249 

5.403 
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